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An  experiment  was  performed  to  assess  the  relative  merits  of  Program  Design 
Languages  (PDLs)  and  flowcharts  as  techniques  for  the  development  and  documenta- 
tion of  detailed  designs  for  computer  programs. 

Twenty  students  in  a computer  science  graduate  course  participated  in  this 
experiment.  Working  individually,  the  students  designed  a two-pass  assembler 
for  a simple  minicomputer-.  Half  the  students  expressed  their  design  *>> — /l 
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for  the  first  pass  of  the  assembler  in  the  form  of  a flowchart,  and  expressed 
their  design  for  the  second  pass  in  a Program  Design  Language.  The  other 
half  of  the  students  used  a PDL  for  pass  one,  and  a flowchart  for  pass  two. 
Flowcharts  and  PDLs  were  compared  on  the  basis  of  various  measures  of  overall 
design  quality,  design  errors,  level  of  detail  of  designs,  time  expended  in 
developing  designs,  and  subjective  preferences.^ 

Having  completed  this  design  task,  the  subjects  then  performed  an  implementation 
task.  They  were  given  fairly  detailed  procedural  designs  for  a program  which 
simulates  the  function  of  a fairly  sophisticated  minicomputer.  They  were  then 
required  to  develop  a working  version  of  the  program  in  PL/1.  Although  the 
designs  were  logically  equivalent,  half  the  students  received  their  simulator 
design  in  flowchart  form,  and  half  in  PDL  form.  Flowcharts  and  PDLs  were 
compared  on  the  basis  of  design  comprehension  test  performance,  various  measures 
of  overall  implementation  quality,  implementation  errors,  and  subjective  pre- 
ferences. 

In  the  context  in  which  this  study  was  performed,  the  use  of  a Program  Design 
Language  (PDL)  by  a software  designer,  for  the  development  and  description  of 
a detailed  program  design,  produced  better  results  than  did  the  use  of  flow- 
charts. Specifically,  the  designs  appeared  to  be  of  significantly  better 
quality,  involving  more  algorithmic  or  procedural  detail,  than  those  produced 
using  flowcharts.  In  addition,  flowchart  designs  exhibited  considerably  more 
abbreviation  and  other  space-saving  practices  than  did  PDL  designs,  with  a 
possible  adverse  effect  on  their  readability. 

When  equivalent,  highly  readable  designs  were  presented  to  subjects  in  both  PDL 
and  flowchart  form,  no  pattern  of  short-term  or  long-term  differences  in  com- 
prehension of  the  design  was  observed.  No  significant  differences  were 
detected  in  the  quality  or  other  properties  of  programs  written  as  implemen- 
tations of  the  designs.  Subjective  ratings  indicated  a mild  preference  for  PDLs. 

Overall,  the  results  suggest  that  software  design  performance  and  designer- 
programmer  communication  might  be  significantly  improved  by  the  adoption  of 
Informal  Program  Design  Languages,  rather  than  flowcharts,  as  a standard 
documentation  method  for  detailed  computer  program  designs. 
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FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  is  concerned  with 
the  human  resource  demands  of  increasingly  complex  battlefield  systems 
used  to  acquire,  transmit,  process,  disseminate,  and  utilize  information. 
This  increased  complexity  places  greater  demands  upon  the  operator  inter- 
acting with  the  machine  system.  Research  in  this  area  is  focused  on 
human  performance  problems  related  to  interactions  within  command  and 
control  centers  as  well  as  issues  of  system  development. 

It  is  concerned  with  such  areas  as  software  development,  topographic 
products  and  procedures,  tactical  symbology,  user  oriented  systems,  infor- 
mation management,  staff  operations  and  procedures,  and  sensor  systems 
integration  and  utilization. 

One  area  of  special  interest  involves  the  development  of  computer 
software  to  support  automated  battlefield  systems.  Software  development 
is  a costly,  unreliable,  not  well  understood  process.  The  present 
research  assessed  the  relative  merits  of  Program  Design  Languages  (PDLs) 
and  flowcharts  as  techniques  for  the  development  and  documentation  of 
detailed  designs  for  computer  programs.  The  research  is  part  of  a larger 
effort  to  develop  a conceptualization  of  the  programming  process  and 
identify  behavioral  bottlenecks  in  software  development.  Efforts  in  this 
area  are  directed  at  improving  accuracy  and  productivity  in  programming 
through  the  design  of  procedures,  languages,  and  methods  to  enhance 
programmer  performance. 

Research  in  the  area  of  human  factors  in  software  development  is 
conducted  as  an  in-house  effort  augmented  contractually  by  organizations 
selected  as  having  unique  capabilities  and  facilities.  The  effort  is 
responsive  to  requirements  of  Army  Project  2Q762725A778,  and  to  general 
requirements  expressed  by  members  of  the  Integrated  Software  Research  and 
Development  Working  Group  (ISRAD). 


^ »-  V '■■■  - 


BRIEF 


Requirement: 

This  experimental  study  was  performed  in  an  attempt  to  assess  the 
relative  merits  of  Program  Design  Languages  (PDLs)  and  flowcharts  as  tech- 
niques for  the  development  and  documentation  of  detailed  designs  for  com- 
puter programs. 

Procedure: 

< 

Twenty  students  in  a computer  science  graduate  course  participated 
in  an  experiment.  Working  individually,  the  students  designed  a two-pass 
assembler  for  a simple  minicomputer.  Half  the  students  expressed  their 
design  for  the  first  pass  of  the  assembler  in  the  form  of  a flowchart, 
and  expressed  their  design  for  the  second  pass  in  a Program  Design  Language. 
The  other  half  of  the  students  used  a PDL  for  pass  one,  and  a flowchart 
for  pass  two.  Flowcharts  and  PDLs  were  compared  on  the  basis  of  various 
measures  of  overall  design  quality,  design  errors,  level  of  detail  of 
designs,  time  expended  in  developing  designs,  and  subjective  preferences. 

Having  completed  this  design  task,  the  subjects  then  performed  an 
Implementation  task.  They  were  given  fairly  detailed  procedural  designs 
for  a program  which  simulates  the  function  of  a fairly  sophisticated 
minicomputer.  They  were  then  required  to  develop  a working  version  of  the 
program  In  PL/1.  Although  the  designs  were  logically  equivalent,  half  the 
students  received  their  simulator  design  in  flowchart  form,  and  half  in 
PDL  form.  Flowcharts  and  PDLs  were  compared  on  the  basis  of  design  com- 
prehension test  performance,  various  measures  of  overall  implementation 
quality,  implementation  errors,  and  subjective  preferences. 
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Findings: 


In  the  context  in  which  this  study  was  performed,  the  use  of  a 
Program  Design  Language  (PDL)  by  a software  designer,  for  the  development 
and  description  of  a detailed  program  design,  produced  better  results  than 
did  the  use  of  flowcharts.  Specifically,  the  designs  appeared  to  be  of 
significantly  better  quality,  involving  more  algorithmic  or  procedural 
detail,  than  those  produced  using  flowcharts.  In  addition,  flowchart 
designs  exhibited  considerably  more  abbreviation  and  other  space-saving 
practices  than  did  PDL  designs,  with  a possible  adverse  effect  on  their 
readability. 

When  equivalent,  highly  readable  designs  were  presented  to  subjects 
in  both  PDL  and  flowchart  form,  no  pattern  of  short-term  or  long-term 
differences  in  comprehension  of  the  design  was  observed.  No  significant 
differences  were  detected  in  the  quality  or  other  properties  of  programs 
written  as  implementations  of  the  designs.  Subjective  ratings  indicated 
a mild  preference  for  PDLs. 

Utilization  of  Findings: 

Overall,  the  results  suggest  that  software  design  performance  and 
designer-programmer  communication  might  be  significantly  improved  by  the 
adoption  of  informal  Program  Design  Languages,  rather  than  flowcharts, 
as  a standard  documentation  method  for  detailed  computer  program  designs. 
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INTRODUCTION 

In  recent  years,  several  trends  have  emerged  which  have  caused  the 
Department  of  Defense  to  direct  increasing  attention  toward  the  investiga- 
tion of  the  behavioral  factors  which  affect  the  development  of  computer 
software.  DoD  software  expenditures,  which  probably  now  exceed  $5  billion 
annually,  have  been  increasing  both  in  absolute  terms  and  as  a percentage 
of  total  system  development  costs  (Boehm,  1973).  At  the  same  time,  the 
increasing  size  and  complexity  of  software  systems  has  caused  software 
reliability  to  become  a dominating  problem  in  many  modern  weapon  and  sup- 
port systems  (Boehm  & Haile,  1972).  During  the  1970s,  there  has  been  an 
emerging  awareness,  both  in  DoD  and  in  the  computer  science  community  at 
large,  that  the  control  of  software  costs  and  quality  will  depend  ultimately 
on  an  understanding  of  the  complex  behavioral  factors  affecting  software 
design,  programming,  documentation,  etc. 

The  quality  of  software  documentation  can  significantly  influence 
both  the  quality  and  cost  of  the  software  (to  the  extent  that  the  documen- 
tation is  used  during  the  development  effort)  and  its  maintainability  and 
modifiability.  Software  documentation  is  an  area  which  has  long  been  sub- 
ject to  DoD  standards.  A number  of  new  software  documentation  methods  have 
recently  emerged  (e.g.,  HIP0  charts,  structure  charts,  program  design 
languages).  DoD  is  now  considering  recommendations  that  some  of  these 
techniques  be  incorporated  more  widely  in  its  standards.  One  of  these 
techniques  (Program  Design  Languages)  is  the  subject  of  this  study. 

The  importance  of  documentation  seems  to  be  widely  recognized  in 
the  software  development  community.  For  example,  when  programming  managers 
were  asked  to  identify,  via  a two-stage  Delphi  technique,  those  variables 
which  most  affect  programmer  productivity,  documentation  quality  was  rated 
the  most  important  factor  (Scott  & Simmons,  1974).  Yet,  in  spite  of  this 


consensus  about  the  importance  of  good  documentation,  there  is  considerable 
disagreement  concerning  the  form  it  should  take. 

One  of  the  reasons  for  the  wide  diversity  of  documentation  types 
is  the  diversity  of  purposes  for  which  software  documentation  can  be  used. 
Judd  (1972)  advocates  the  development  of  program  design  documentation, 
error  diagnosis  documentation,  operational  documentation,  maintenance 
documentation,  development  documentation,  and  marketing  documentation, 
all  as  separate  products  of  a software  development  effort.  It  is  undoubtedly 
true  that  documentation  methods  appropriate  for  some  of  these  purposes 
may  not  be  appropriate  for  others.  This  report  focuses  upon  "detailed 
program  design"  documentation,  typically  developed  by  a software  system 
designer  and  given  to  a programmer  as  the  specification  of  his  programming 
task.  It  is  the  formal  medium  of  communication  between  the  designer  and 
the  programmer. 

Flowcharts  have  long  been  accepted  in  the  computer  science  com- 
munity as  the  standard  medium  for  detailed  program  design  documentation, 
as  well  as  for  archival  documentation  for  later  use  by  maintenance  pro- 
grammers. Recently,  however,  the  use  of  flowcharts  for  archival  documenta- 
tion has  been  questioned  by  a number  of  critics,  who  suggest  that  flow- 
charts may  not  aid  program  comprehension  (Weinberg,  1971),  or  error  diagnosis 
(Aron,  1974),  and  that  they  are  an  unnecessary  drain  on  project  resources 
(Brooks,  1975).  White  (1975)  has  suggested  that  modern  programming  languages 
provide  a more  powerful  and  concise  documentation  medium,  in  their  own 
right,  than  do  flowcharts  which  attempt  to  convey  the  procedural  details 
of  a program. 

Shneiderman  et  al.  (1977)  performed  a series  of  experiments  to 
determine  whether  detailed  flowcharts  might  improve  the  programmer's 
performance  in  any  of  several  programming  tasks.  Flowcharting  before 
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coding  did  not  improve  the  quality  of  the  resulting  program,  as  graded 
by  a graduate  assistant.  When  programs  were  presented  with  or  without 
flowcharts,  flowcharts  did  not  result  in  significantly  improved  perfor- 
mance on  either  comprehension  tests  or  modification  tasks.  In  one  study, 
comprehension  even  appeared  to  be  degraded  by  the  presence  of  a flowchart. 

In  a study  in  which  some  subjects  had  prior  experience  with  flowcharts 
while  others  did  not,  the  experienced  subjects  performed  somewhat  better 
with  flowcharts,  while  the  inexperienced  subjects  did  not. 

Flowcharts  and  similar  documentation  methods  have  also  been  studied 
outside  the  software  context.  Kammann  (1975)  reported  an  experiment 
involving  a telephone  dialing  task  in  which  subjects  (housewives  and  tech- 
nical personnel)  performed  better  when  it  was  presented  in  the  form  of  a 
flowchart  than  when  presented  in  the  form  of  a text  description.  An 
experiment  by  Wright  and  Reid  (1973)  has  identified  what  may  be  a key  task 
variable  in  these  studies.  In  their  experiment,  subjects  were  required 
to  memorize  a procedure  or  to  execute  the  procedure  by  hand  without 
necessarily  memorizing  it.  When  the  subjects  hand-executed  the  procedure 
while  its  description  was  still  available  to  them,  performance  was  better 
for  procedures  described  by  flowchart  or  decision  table  than  for  those 
described  by  prose  or  a series  of  "short  sentences."  But  when  the  sub- 
ject's task  was  execution  of  the  procedure  from  memory,  "short  sentences" 
were  best,  and  performance  with  flowcharts  was  rather  poor.  The  authors 
suggest  that  rehearsal  effects  may  account  for  this  difference,  since 
material  presented  verbally  may  be  more  readily  rehearsed. 

Wright  and  Reid  suggest  that  information  presented  in  flowchart 
form  is,  at  least  in  part,  encoded  visually,  with  the  result  that  success- 
ful transfer  to  long-term  memory  is  hindered.  On  the  other  hand,  informa- 
tion presented  in  short-sentence  form  is  encoded  verbally  and  is  more  likely 
to  be  successfully  integrated  into  long-term  memory.  The  mechanism  suggest- 
ed by  Wright  and  Reid  (dual  encoding  with  differential  rehearsal  effects) 
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seems  plausible  (cf.  Craik,  1973;  Craik  and  Lockhart,  1972;  Paivio, 

1969).  In  any  event,  the  study  certainly  suggests  that  these  two 
information  presentation  forms  have  different  retention  patterns.  It 
should  be  noted,  however,  that  exposure  time  was  limited  in  this  study, 
and  that  the  two  presentation  forms,  though  equal  in  logical  information 
content,  differed  somewhat  in  information  structure. 

In  general  these  results  suggest  that  flowcharts  may  aid  hand- 
execution  of  a procedure,  but  may  not  aid  comprehension  of  the  procedure. 
Shneiderman  et  al.  (1977)  present  some  data  which  support  this  hypothesis 
In  one  study,  they  report  separately  "comprehension"  questions  which 
involve  hand  execution  and  questions  which  involve  interpretation  of 
the  programs.  Subjects  who  were  given  both  flowcharts  and  programs 
did  about  as  well  on  hand-execution  items  as  those  who  were  given  only 
the  programs.  On  interpretation  items,  though,  the  program-only  group 
performed  somewhat  better. 

Unfortunately,  it  may  not  be  legitimate  to  generalize  these 
findings  to  large  software  projects  or  to  summary  design  documentation. 
Those  studies  which  have  addressed  programming  tasks  have  not  only  been 
restricted  to  relatively  inexperienced  student  programmers,  but  have 
also  investigated  only  relatively  small,  simple  programs.  Even  if  the 
program  itself  constitutes  a better  form  of  archival  documentation  than 
does  a flowchart,  the  size  of  a program  may  be  a significant  factor. 
Beyond  a certain  size,  a program(s)  cannot  be  read  directly  by  one  person 
and  summary  documentation  of  some  sort  is,  therefore,  required  in  large 
software  projects.  Whether  flowcharts  or  some  other  form  are  best 
for  this  purpose  is  not  known.  Even  for  programs  of  intermediate  size, 
the  reader  might  be  aided  by  such  summary  documentation. 

Although  the  referenced  studies  have  been  concerned  primarily 
with  archival  documentation,  rather  than  with  designer-to-programmer 


design  communication,  they  may  have  implications  for  the  latter. 

The  data  suggest  that  a computer  program,  expressed  in  a high-level 
programming  language,  is  more  comprehensible  than  the  corresponding 
flowchart.  If  this  is  true,  then  an  artificial  design  language,  with 
a programming-language-like  syntax,  might  also  be  preferable  to  flow- 
charts for  the  expression  of  software  design  information.  Such  languages, 
commonly  called  Program  Design  Languages  (PDLs)  have  been  discussed  in 
the  literature  for  several  years.  PDLs  have  gradually  evolved  from 
the  rather  informal  use  of  "pseudo-code"  to  express  designs  or  algorithms. 
Gradually,  this  form  of  documentation  became  more  formal,  evolving  into 
a PDL  form.  Although  a few  presumably  successful  case  studies  have 
been  reported  without  substantiating  data  (e.g..  Van  Leer,  1976),  the 
authors  are  unaware  of  any  experimental  studies  bearing  directly  on 
the  issue  of  PDL  efficacy. 

Figure  1 is  an  example  of  PDL  specification  for  a program  which 
computes  social  security  withholding  (FICA)  amounts  from  a payroll 
data  base,  and  prints  a report  of  those  values.  This  example  is  taken 
from  a report  (Kraly  et  al.,  1975)  concerning  program  design  methods 
and  documentation  techniques  recommended  for  adoption  by  DoD. 


PDLs  can  vary  greatly  in  the  degree  of  formality.  At  one  extreme, 
is  a "freeform"  PDL,  such  as  that  in  Figure  1.  In  a freeform  PDL,  there 
is  a minimum  of  syntactic  and  semantic  constraints;  the  terms  and  expres- 
sions that  are  used  are  determined  by  the  user.  In  a "formal"  PDL, 
the  syntax  and  semantics  are  highly  constrained;  the  user  is  restricted 
to  using  keywords  (e.g.  READ,  WRITE,  etc.)  and  elements  of  an  expression 
may  be  syntactically  identified  by  underlines,  parentheses,  etc.  In 
general,  a formal  PDL  is  more  precise  than  an  informal  PDL,  but  it  is 
also  more  difficult  to  learn  and  use.  Formal  PDLs  tend  to  be  preferred 
especially  when  the  design  specification  is  subject  to  direct  computer 
processing,  as  for  consistency  checking. 
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PRINT  FICA  REPORT  HEADER 

OBTAIN  FICA  PERCENT  AND  FICA  LIMIT  FROM  CONTRAINTS  FILE 
SET  FICA  TOTAL  TO  ZERO 
DO  FOR  EACH  RECORD  IN  SALARY  FILE 

OBTAIN  EMPLOYEE  NUMBER  AND  TOTAL  SALARY  TO  DATE 
IF  TOTAL  SALARY  IS  LESS  THAN  FICA  LIMIT  THEN 

SET  FICA  VALUE  TO  TOTAL  SALARY  TIMES  FICA  PERCENT 

ELSE 

SET  FICA  VALUE  TO  FICA  LIMIT  TIMES  FICA  PERCENT 
ENDIF 

PRINT  EMPLOYEE  NUMBER  AND  FICA  VALUE 
ADD  FICA  VALUE  TO  FICA  TOTAL 
ENDDO 

PRINT  FICA  TOTAL 

,1 


Figure  1.  An  Example  of  a PDL  Specification 


! 

4 


There  are  several  reasons  for  believing  that  POLs  and  flowcharts 
may  differentially  affect  performance  in  design,  design  documentation, 
and  design  comprehension  tasks.  First,  information  presented  in  these 
two  media  may  be  encoded  in  memory  in  different  ways,  at  least  with 
limited  exposure  time.  Notice  that  PDL  design  specifications  are  very 
similar  to  the  "short  sentences"  used  by  Wright  and  Reid  (1973), 
while  flowcharts  are  similar  to  their  "algorithms." 

Secondly,  POL  and  flowchart  forms  may  differ  in  the  processing 
effort  required  to  encode  them  in  memory  even  if  they  are  encoded 
similarly.  PDL  forms  are  not  only  already  verbal  (and  thus,  perhaps, 
more  easily  integrable  into  long-term  memory),  but  also  appear  to  be 
more  compatible  with  the  hierarchic  propositional  memory  representation 
hypothesized  by  Atwood  and  Ramsey  (in  press)  for  program  comprehension. 
Program  documentation  which  has  a similar  structure  to  the  representa- 
tion of  the  program  in  memory  may  require  less  processing  in  order  to 
be  encoded  in  memory.  Given  sufficient  processing  time,  however, 
flowchart  information  might  be  encoded  in  the  same  way  as  PDL  information* 
even  if  they  are  encoded  differently  with  brief  exposure  times,  as 
suggested  by  Wright  and  Reid  in  (1973)  results.  In  most  actual  software 
development  situations,  exposure  time  is  not  highly  constrained;  thus, 
if  this  is  the  only  difference  between  PDLs  and  flowcharts  from  the  point 
of  view  of  comprehension,  it  may  result  in  only  a mild  preference  for 
one  technique  over  the  other. 

Third,  PDLs  and  flowcharts  may  emphasize  different  properties 
of  the  underlying  software  design.  At  an  obvious  level,  flowcharts 
appear  to  emphasize  flow  of  control,  while  PDLs  may  have  a greater 
emphasis  on  program  structure.  This  may  affect  the  performance  of  the 
designer,  as  well  as  the  programmer.  From  the  designer's  viewpoint,  it 
may  be  that  different  classes  of  design  problems  are  more  easily  detect- 
able with  different  design  documentation  formats.  More  basically,  it 
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may  simply  be  easier  to  express  the  properties  which  the  designer 
considers  important  in  one  way  than  in  the  other.  It  is  even  conceiv- 
able that  some  types  of  design  are  more  amenable  to  PDLs,  while  others 
are  more  compatible  with  flowcharts. 

From  the  viewpoint  of  the  programmer,  the  form  in  which  the 
programming  problem  is  expressed  may  very  well  affect  his  basic  under- 
standing of  the  problem.  In  various  problem-solving  tasks,  it  has  been 
shown  that  the  perceptual  process  is  important  to  the  formation  of 
initial  conceptual  problem  structures  (Simon  & Barenfield,  1969),  and 
that  the  form  in  which  a problem  is  expressed  can  thereby  affect  problem 
solving  strategy  (Simon  & Hayes,  1976)  and  performance  (Jeffries  et  al , 
1977;  Simon  & Hayes,  1976). 

Fourth,  and  finally,  it  seems  fairly  clear  that  PDLs  have  some 
practical  advantages,  such  as  decreased  production  costs  and  easier 
maintainability,  relative  to  flowcharts  and  other  graphical  techniques 
(Ortega,  1974). 


Thus,  an  analytical  comparison  of  PDLs  and  flowcharts  would 
appear,  overall,  to  favor  PDLs  for  detailed  design  documentation. 

Only  empirical  evaluation,  however,  can  provide  really  convincing  evid- 
ence in  favor  of  one  or  another  technique.  The  following  sections 
will  present  an  experimental  study  performed  to  obtain  such  evidence. 

The  experiment  reported  here  was  intended  to  provide  empirical 
evidence  which  might  assist  in  the  selection  of  flowcharts  or  PDLs  as  a 
medium  for  detailed  program  design  documentation.  The  experiment  was 
intended  to  investigate  the  effects  of  documentation  format  both  on 
performance  of  the  designer  in  a software  design  task  and  on  performance 
of  the  programmer  in  a program  implementation  task.  Although  the  study 
was  confined  to  one-person  design  and  implementation  efforts,  the  tasks 
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(especially  the  implementation  task)  were  much  larger  than  those  which 
have  typically  been  employed  in  behavioral  studies  of  software  develop- 
ment, and  were  fairly  "realistic"  in  terms  of  both  program  size  and 
programmer  experience.  The  study  attempted  to  observe  the  effects  of 
documentation  format  on  design  quality,  detailed  design  properties, 
designer  effort,  design  comprehension  by  the  programmer,  resulting 
program  quality  and  error  properties,  programmer  effort,  and  subjective 
preferences  of  the  participants. 
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METHOD 


OVERVIEW 

Twenty  students  in  a computer  science  graduate  course  participated 
in  the  experiment.  Working  individually,  the  students  designed  a two- 
pass  assembler  for  a simple  minicomputer.  Half  the  students  expressed 
their  design  for  the  first  pass  of  the  assembler  in  the  form  of  a flow- 
chart, and  expressed  their  design  for  the  second  pass  in  a PDL.  The  other 
half  of  the  students  used  a PDL  for  pass  one,  and  a flowchart  for  pass 
two.  This  2X2  repeated-measures  Latin  square  design  was  employed  in  an 
effort  to  correct  for  individual  differences  and  to  increase  the  statis- 
tical power  which  could  be  achieved  with  the  limited  number  of  subjects 
available.  Although  this  design  has  the  disadvantage  that  any  chance 
group  difference  is  confounded  with  interactions,  it  was  felt  that  the 
advantages  outweighed  this  disadvantage.  Flowcharts  and  PDLs  were  com- 
pared on  the  basis  of  various  measures  of  overall  design  quality,  design 
errors,  level  of  detail  of  designs,  time  expended  in  developing  designs, 
and  subjective  preferences. 

Having  completed  this  design  task,  the  subjects  then  performed  an 
implementation  task.  They  were  given  fairly  detailed  procedural  designs 


for  a program  which  simulates  the  function  of  a fairly  sophisticated 
minicomputer.  They  were  then  required  to  develop  a working  version  of  the 
program  in  PL/1.  Although  the  designs  were  logically  equivalent,  half 
the  students  received  their  simulator  design  in  flowchart  form,  and  half 
in  PDL  form.  Subjects  from  each  design-phase  group  were  divided  equally 
into  these  two  implementation-phase  groups.  Flowcharts  and  PDLs  were 
compared  on  the  basis  of  design  comprehension  test  performance,  various 
measures  of  overall  Implementation  quality,  implementation  errors,  time 
expended  in  developing  implementation,  and  subjective  preferences. 
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SUBJECTS 
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Subjects  were  1 female  and  19  male  graduate  students  at  Oklahoma 
State  University  (OSU).  With  one  exception,  subjects  were  Master  of 
Science  candidates  majoring  in  Computer  Science  and  enrolled  in  the 
course,  "Computer  Structure  and  Programming,"  described  below.  One 
student,  who  later  withdrew  from  the  course,  was  majoring  in  engineering. 
Because  of  the  location  of  OSU  away  from  large  urban  centers,  and  pos- 
sibly other  factors,  students  in  this  curriculum  tend  to  be  full-time 
students  with  no  outside  (non- university)  employment.  Subjects  ranged 
in  age  from  21  to  40,  with  a mean  age  of  26.1  years.  They  had  completed, 
on  the  average,  8.9  computer  science  courses  (undergraduate  and/or 
graduate)  in  addition  to  current  enrollment  (range  4-16). 

Based  on  previous  knowledge  of  the  faculty,  two  subject  factors 
were  identified  as  sufficiently  relevant  to  the  subjects'  performance 
in  the  study  to  warrant  a special  effort  to  balance  the  effects  of  these 
factors  across  experimental  groups.  These  factors  were  native  language 
(a  significant  number  of  foreign  students  attend  OSU)  and  participation 
in  a particular  graduate  student  curriculum,  discussed  below.  Five  of 
the  subjects  were  not  native  speakers  of  English.  Two  of  these  five 
were  considered  completely  fluent  in  English  by  their  instructor.  The 
remaining  three  were  considered  competent  in  English;  all  three  had 
taken  prior  degrees  at  English-language  universities.  As  a precautionary 
measure,  however,  these  three  subjects  were  assigned  (at  random)  to 
different  experimental  groups. 

Three  of  the  subjects  were  pursuing  a specialized  curriculum 
(the  "Programming  Language  Option"),  which  emphasizes  compilers,  as- 
semblers, and  machine  simulators.  Since  the  design  and  programming 
problems  used  in  the  experiment  involved  an  assembler  and  a machine 
simulator,  these  three  subjects  were  assigned  (at  random)  to  different 
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experimental  groups.  With  the  two  exceptions  just  described,  subject 
assignment  to  experimental  groups  was  entirely  random. 

The  particular  course  in  which  the  experiment  was  performed  was 
entitled,  "Computer  Structure  and  Programming."  As  described  in  the 
department's  brochure,  the  course  covers  the  areas  of: 


Computer  structure,  flow  of  control,  arithmetic  and  logical 
operations.  Machine  instructions,  indexing,  addressing, 
linkages  and  input-output.  Assembly,  macro  and  emulation- 
simulation  systems. 


This  particular  required  course  has  a reputation,  among  the  students, 
of  being  the  most  difficult  course  in  the  Master's  curriculum. 


PRELIMINARY  PHASE 


The  first  phase  of  the  experiment  was  introductory  in  nature  and 
involved  the  same  procedures  for  all  students.  The  experiment  began  at 
a class  session  approximately  3 weeks  after  the  start  of  the  course  (a 
detailed  schedule  of  events  can  be  found  in  Appendix  A).  In  this  class 
session,  the  students  were  given  an  introduction  to  the  experiment,  as 
well  as  an  introduction  to  the  concept  of  Program  Design  Languages,  and 
were  asked  to  provide  background  information  describing  their  computer 
science  experience  and  some  personal  characteristics. 

The  lecture  began  with  a brief  discussion  of  the  nature  of  the 
experiment.  A concerted  effort  was  made  to  introduce  the  experiment  in 
an  unbiased  way.  Statements  of  specific  hypotheses  were  avoided,  and 
the  experiment  was  described  as  an  attempt  to  compare  two  techniques, 
each  of  which  probably  has  both  advantages  and  disadvantages.  It  was 
pointed  out  that  participation  was  entirely  voluntary,  that  the  experi- 
ment was  entirely  consistent  with  course  goals  and  would  involve  little 
additional  effort  by  the  subjects,  and  that  subjects  completing  all 
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phases  of  the  experiment  would  be  paid  $25.  A more  detailed  impression 
of  the  way  in  which  the  experiment  was  described  to  the  students  can  be 
gained  by  reading  the  consent  form  which  they  were  given  (see  Appendix  B). 

Because  flowcharts  were  the  standard  mechanism  for  the  procedural 
description  of  programs  in  OSU  courses  prior  to  this  experiment,  most  of 
the  subjects  had  had  little  prior  exposure  to  PDLs.  Thus,  it  was  neces- 
sary to  provide  them  with  sufficient  training  in  the  use  of  PDLs  that 
they  would  be  able  to  generate  and  comprehend  specifications  expressed 
in  this  form.  At  the  same  time,  it  was  obviously  desirable  to  minimize 
the  biasing  effect  that  might  result  from  a training  session  devoted 
purely  to  a discussion  of  PDLs.  We  attempted  to  satisfy  both  of  these 
objectives  by  presenting  parallel  discussions  of  PDLs  and  flowcharts. 

After  the  basic  concept  of  a Program  Design  Language  had  been  introduced, 
further  discussion  was  directed  primarily  toward  illustrating  how  a 
particular  construct  might  be  expressed,  both  in  PDL  and  in  flowchart 
form.  This  presentation  took  the  form  of  a discussion  of  two  handouts 
(Appendices  D and  E)  which  were  given  to  the  students  at  the  start  of 
the  class  session.  The  discussion  was  recorded  on  audio  tape  for  later 
reference. 

Appendix  D served  the  dual  functions  of  introducing  the  concept 
of  a PDL  to  the  student  and  of  defining  the  particular  PDL  (and  flowchart) 
conventions  which  were  to  be  used  in  both  the  design  and  implementation 
stages  of  the  study.  The  particular  PDL  used  was  a relatively  unstruc- 
tured or  "freeform"  PDL  (cf.  Kraly  et  al,  1975).  That  is,  the  user  was 
required  to  use  a particular  basic  syntax  involving  a particular  set  of 
basic  structural  constructs  (PROCEDURE. . .END,  IF. . .THEN. . .ELSE,  DO... END, 
etc.),  but  was  relatively  unconstrained  with  respect  to  the  particular 
statements  which  might  appear  within  these  basic  structures.  He  might 
say  INCREMENT  ITERATION_COUNT,  ADD  ONE  TO  ITERATIONjCOUNT , UPDATE 
ITERATION  COUNT,  ITERATION  COUNT  = ITERATION  COUNT  +1,  etc.  The 
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particular  PDL  used  in  the  study  was  chosen  to  correspond  in  many  respects 
to  features  of  PL/1,  with  which  students  were  familiar.  Similarly,  the 
basic  set  of  flowchart  symbols  available  to  the  user  was  specified,  and 
consisted  of  the  flowchart  symbols  in  general  use  in  OSU  computer  science 
graduate  courses.  The  particular  statements  which  might  appear  within 
flowchart  symbols  were  relatively  unconstrained,  however. 

The  second  handout  (Appendix  E)  contained  the  specification  for  a 
very  simple  minicomputer  simulator,  expressed  in  both  flowchart  and  PDL 
form.  This  handout,  and  its  discussion,  served  not  only  to  provide  a 
concrete  example  of  both  flowchart  and  PDL  specifications,  but  also  to 
Introduce  the  students  to  some  of  the  concepts  involved  in  machine  simu- 
lators. 

When  the  PDL-flowchart  presentation  had  been  completed,  each 
student  was  given  two  consent  forms  (Appendix  B)  and  a background  question 
naire  (Appendix  C).  The  students  were  asked  to  sign  the  consent  forms, 
fill  out  the  questionnaire,  and  return  them  at  the  following  class 
session,  if  they  wished  to  participate  in  the  experiment.  All  20  students 
agreed  to  participate,  and  returned  all  of  the  requested  forms. 


DESIGN  PHASE 


In  the  design  phase  of  the  experiment,  the  subjects  were  given 
the  task  of  designing  a two-pass  assembler  for  a minicomputer.  The 
particular  minicomputer  involved  was  a hypothetical  machine  (the  "small 
computer"  which  had  been  defined  and  used  as  an  example  earlier  in  the 
course.  The  subjects  were  presumably  thoroughly  familiar  with  the  hard- 
ware instruction  set  of  this  machine.  Earlier  in  the  course  they  had 
been  instructed  in  the  use  of  the  programming  language  APL  as  a notation 
for  concise,  rigorous  definition  of  hardware  instruction  sets.  As  an 
exercise,  they  had  then  developed  a complete  specification,  in  APL,  of 
the  instruction  set  of  this  particular  machine. 


In  addition  to  this  familiarization  with  the  machine,  the  students 
had  been  given  instructions  concerning  the  design  of  assemblers  as  part 
of  the  course  in  which  the  experiment  was  conducted.  No  prior  discussion 
had  occurred  with  respect  to  the  possible  properties  of  an  assembler,  or 
of  an  assembler  language,  for  the  "small  computer,"  however.  At  the 
start  of  the  design  phase  of  the  experiment,  the  subjects  were  given  a 
handout  (Appendix  F)  which  described  this  assembler  language,  and  which 
instructed  the  students  that  they  would  be  required  to  design  a two-pass 
assembler,  documenting  one  pass  via  flowchart  and  the  other  pass  via  POL. 

Along  with  the  general  assembler  design  problem  description,  the 
subjects  received  a brief  description  of  the  responsibilities  of  the  first 
pass  of  the  assembler  (Appendix  G).  These  two  handouts  constituted  the 
entire  problem  description  for  pass  1.  All  subjects  designed  the  pass  1 
program  first,  but  half  of  the  subjects  were  told  to  prepare  their  de- 
signs in  flowchart  form,  and  half  in  PDL  form.  A third  handout  (Appendix 
I)  accompanied  the  problem  description.  This  was  a form  on  which  the 
subjects  were  asked  to  record  the  amount  of  time  expended  on  the  project 
in  each  of  several  categories  (initial  design  preparation,  design 
revision,  hand-checking,  preparation  of  documentation,  etc.).  The 
subjects  were  told  to  prepare  their  designs  and  turn  them  in  with  the 
time  questionnaire,  whenever  they  were  completed,  at  which  time  they 
would  be  given  pass  2 instructions.  They  were  told  that  they  had  at 
most  five  days  to  complete  the  first  pass,  and  that  the  second  pass 
design  would  be  due  seven  days  after  the  start  of  the  design  phase. 

When  a subject  turned  in  his  pass  1 design,  he  was  given  a descrip- 
tion of  the  responsibilities  of  the  pass  2 program  (Appendix  H),  along 
with  a time  questionnaire  essentially  identical  to  that  shown  in 
Appendix  I.  Those  subjects  who  had  prepared  their  pass  1 designs  in 
flowchart  form  were  instructed  to  use  the  POL  form  for  pass  2,  and  vice- 


versa. 


After  completion  of  the  pass  2 designs,  subjects  were  given  a 
take-home  questionnaire  (Appendix  J)  which  asked  them  to  subjectively 
compare  flowcharts  and  PDLs  with  respect  to  their  overall  utility  to 
the  software  designed  and  to  the  programmer,  and  a number  of  more 
detailed  criteria. 

In  addition  to  the  time  and  subjective  preference  data,  finished 
designs  were  graded  on  overall  quality  by  an  expert  judge,  and  were 
analyzed  for  the  nature  of  design  errors,  level  of  detail  of  the  design, 
and  other  properties.  This  judge  was  an  associate  professor  on  the 
computer  science  department  faculty,  very  knowledgable  about  software 
design  techniques  and  methods,  and  highly  experienced  in  developing 
assemblers  and  translators. 

IMPLEMENTATION  PHASE 


In  the  implementation  phase,  the  subjects  were  given  a program 
design,  expressed  in  either  flowchart  or  POL  form,  and  were  required  to 
develop  a working  version  of  the  program  in  PL/1.  This  implementation 
effort  was  the  major  term  project  for  this  course.  The  program  was  a 
simulator  for  the  Data  General  MicroNova  minicomputer.  As  minicomputers 
go,  the  instruction  set  and  execution  behavior  of  the  MicroNova  are 
fairly  complicated.  The  simulator,  which  was  to  run  on  an  IBM  360,  was 
intended  to  correctly  execute  MicroNova  machine  language  programs,  with 
the  exception  that  console  operations  were  excluded  and  input/output 
device  control  was  somewhat  simplified.  In  addition,  the  simulator 
design  included  some  specialized  capabilities  (Trace,  Dump,  etc.)  which 
were  associated  with  the  simulator  itself. 

Prior  to  the  start  of  the  experimental  portion  of  the  implementa- 
tion phase,  the  students  studied  the  properties  of  the  MicroNova,  in  order 
to  gain  a thorough  understanding  of  the  machine  itself.  This  study  in- 
cluded lectures,  reading  of  MicroNova  reference  documents  (e.g.,  Data 
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General  Corp.,  1976),  and,  in  some  cases,  additional  literature  research 
initiated  by  the  students.  Thus,  at  the  time  the  simulator  designs  were 
distributed,  the  students  were  presumably  thoroughly  familiar  with  the 
MicroNova,  but  had  no  knowledge  of  the  detailed  simulator  design,  which 
described  the  program  they  were  to  implement. 

Half  the  subjects  were  given  a simulator  design  expressed  in 
flowchart  form  (Appendix  K),  and  half  were  given  a design  in  PDL  form 
(Appendix  L).  Although  several  minor  design  errors  were  uncovered  later 
and  were  corrected  by  issuing  an  errata  sheet  (Appendix  M),  those  errors 
were  essentially  equivalent  in  the  two  documentation  forms.  The  errata 
sheet  was  issued  12  days  after  distribution  of  the  designs.  A strong 
effort  was  made  to  ensure  complete  logical  equivalence  of  the  two 
versions  of  the  simulator  design,  and  only  one  minor  logical  difference 

m 

between  the  two  versions  has  subsequently  been  detected.  This  difference 
was  discovered  and  corrected  42  days  after  distribution  of  the  designs. 


1)  ' 


In  order  to  control  for  any  residual  differential  effect  of  the 
earlier  ("design  phase")  study  which  might  bias  the  subjects'  performance 
in  this  second  experiment,  the  subjects  from  each  of  the  earlier  groups 
were  divided  equally  between  the  two  experimental  groups  used  in  this 
phase  of  the  study.  Considered  overall,  then,  there  were  four  separate 
groups  of  5 subjects  each  (ignoring  attrition),  which  were  exposed  to 
the  experimental  conditions  illustrated  in  Table  1.  Although  all  20 
subjects  successfully  completed  the  design  stage  of  the  experiment,  4 
subjects  withdrew  from  the  course,  or  withdrew  altogether  from  school, 
and  therefore  failed  to  complete  the  implementation  phase,  as  indicated  in 
the  table. 

Simulator  designs  were  handed  out  at  the  beginning  of  a class 
session.  The  subjects  were  instructed  to  study  the  designs  for  45  minutes 
in  preparation  for  a comprehension  test.  Then,  in  an  attempt  to  determine 
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TABLE  1.  Summary  of  Experimental  Conditions 


whether  PDLs  and  flowcharts  have  any  detectable  differential  effect  on 
early  design  comprehension,  a comprehension  test  (Appendix  N)  was  ad- 
ministered. The  subjects  were  allowed  35  minutes  to  complete  this 
test.  The  test  contained  a variety  of  items  intended  to  provide  measures 
of  several  comprehension  factors: 

• understanding  of  MicroNova  function 

• ability  to  distinguish  between  MicroNova  and  simulator  functions 

• comprehension  of  modular  structure  of  simulator  design 

• comprehension  of  procedural  operation  of  simulator 

• ability  to  pinpoint  probable  modular  location  of  error  causing 

specified  simulator  malfunction 

• ability  to  specify  changes  needed  to  cause  a specified  change 

in  simulator  function. 

(A  second  administration  of  this  test,  which  was  to  have  occurred  after 
one  week  of  study  of  the  designs  by  the  subjects,  was  inadvertently 
delayed  until  after  the  subjects  had  completed  their  program  implemen- 
tation task.) 

After  the  designs  were  distributed,  the  subjects  were  given  8 1/2 
weeks  to  develop  a complete,  working  simulator.  They  were  allowed  to 
establish  their  own  schedules  for  completing  this  task.  Each  subject 
was  required  to  maintain,  and  turn  in  at  the  end  of  the  project,  a pro- 
ject folder.  The  folder  contained  the  entire  printout  of  each  computer 
run  made  in  connection  with  the  project.  Whenever  new  source  programs 
were  compiled,  listings  of  those  programs  appear  in  these  printouts, 
along  with  any  error  messages  issued  by  the  compiler,  link/editor,  etc. 
Input  and  output  of  simulator  test  runs  were  also  included,  as  was  the 
computer's  job  summary  for  each  run.  These  printouts  thus  provide  data 
for  determination  of  simple  summary  statistics,  such  as  number  of  runs, 

CPU  utilization,  etc.,  as  well  as  more  substantive  analyses  of  error 
types  and  locations,  etc. 


r 


Each  project  folder  also  contained  a "run  log,"  in  which  the 
students  recorded  the  purpose  of  each  run,  nature  of  errors  encountered, 
etc.  This  is  a fairly  standard  practice  at  OSU,  and  was  included  in  this 
project  as  an  aid  to  later  detailed  analysis  of  the  individual  runs. 
Finally,  the  project  folders  contained  a weekly  time  questionnaire 
(Appendix  0)  on  which  subjects  indicated,  by  category,  the  time  they 
had  expended  on  the  project  in  the  preceding  week. 


In  addition  to  the  project  folder,  subjects  turned  in,  at  the  end 
of  the  project,  their  finished  simulator  source  programs  in  computer- 
readable  form.  A very  detailed  test  procedure  was  run  on  each  of  these 
programs  in  an  effort  to  assess  correctness  of  simulator  function  and  to 
detect  those  kinds  of  conceptual  or  semantic  errors  which  the  experimenters 
were  able  to  anticipate  a priori. 

After  the  subjects  had  completed  the  implementation  project  and 
turned  in  their  final  programs  and  project  folders,  the  simulator  design 
comprehension  test  (Appendix  N)  and  the  PDL-Flowchart  preference  question- 
naire (Appendix  J)  was  readministered. 


In  addition  to  the  various  qualitative  and  quantitative  measures 
already  mentioned,  the  final  simulator  programs  were  rated  on  overall 
quality  by  an  expert  judge,  (previously  described)  who  also  performed 
fairly  detailed  error  analyses  of  the  final  programs. 
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RESULTS 

DESIGN  PHASE 

The  allocation  to  experimental  groups  of  non-native-Engl ish- 
speakers  and  of  students  in  a specialized  programming-language  curriculum 
was  explicity  constrained.  These  special  inter-subject  differences  were 
dichotomous  and  affected  small  numbers  of  students,  so  that  the  assign- 
ment of  two  such  subjects  to  a single  group  might  significantly  bias  the 
study.  Other  differences  were  assumed  to  be  controlled  by  the  otherwise 
random  subject-to-group  assignment.  Unfortunately,  this  random  assign- 
ment resulted  in  a difference  in  average  ability  between  the  two  groups 
in  the  design  phase  of  the  study.  This  difference  is  reflected,  for 
example,  in  their  Grade  Point  Averages  (GPAs)  for  all  computer  science 
graduate  courses. 


There  is  at  least  a suggestion,  in  the  available  data,  that  this 
group  difference  may  be  related  more  to  ability  than  to  experience.  The 
two  groups  are  almost  identical  with  respect  to  average  number  of  computer 
science  courses  taken  (8.7  for  the  group  designing  the  first  pass  using 
a PDL  versus  9.1  for  the  other  group),  average  number  of  graduate  computer 
science  courses  taken  (4.2  versus  3.5),  and  number  reporting  past  experience 
with  translator  implementation  and  related  tasks  (5  each). 


For  convenience  of  reference,  let  us  refer  to  the  group  which  used 
a PDL  for  pass  1 of  the  assembler,  and  flowcharts  for  pass  2,  as  the  PDL- 
FL  group.  The  other  group  will  be  referred  to  as  the  FL-PDL  group.  Then 
the  mean  GPA  for  the  PDL-FL  group  was  2.986,  while  the  FL-PDL  mean  was 
3.587.  Standard  deviations  were  .606  and  .334,  respectively.  Statistical 
tests  indicate  significant  differences  in  both  the  means  (t( 15)  = 2.71, 
p < .02)  and  variances  (F(9,8)  = 3.30,  p < .10).  The  PDL-FL  group  included 


several  students  whose  overall  performance  was  relatively  low,  both  within 
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this  course  and  in  graduate  school  in  general,  and  who  had  no  counterparts 
in  the  FL-PDL  group,  thus  accounting  for  both  differences. 

This  group  difference  does  not  greatly  interfere  with  our  ability 
to  analyze  the  study  results  and  draw  reasonable  conclusions.  No  such 
difference  occurred  between  experimental  groups  in  the  implementation 
phase  of  the  study.  The  use  of  a Latin  square  design  in  the  design  phase 
of  the  study  sharply  reduces  the  impact  of  the  group  difference  on  the 
analysis  of  that  phase.  In  fact,  the  Latin-square  analysis  of  variance 
employed  for  most  of  the  design-phase  analyses  is  conservative  in  this 
situation  (McNenwu'.  1969,  p.  387).  The  group  difference  should  be  kept 
in  mind  by  the  reader,  however.  If  either  flowchart  or  PDL  is  consistently 
preferable  in  the  design  of  both  pass  1 and  pass  2 of  the  assembler,  such 
a difference  can  be  detected  by  statistical  analysis.  However,  if  for 
some  reason  a PDL  is  better  for  one  pass  and  flowcharting  for  the  other, 
perhaps  due  to  differences  in  the  design  tasks,  that  interaction  is 
confounded  with  the  group  differences  in  this  study  and  cannot  be  detected. 
Based  on  a careful  consideration  of  the  design  tasks  in  passes  1 and  2,  no 
such  interaction  was  expected. 

Time  to  Produce  Designs 

Table  2 presents  a summary  of  the  students'  responses  to  the  design 
phase  time  expenditure  questionnaire.  On  the  average,  the  students  re- 
ported spending  about  7.6  hours  on  the  pass  1 design  task  and  about  5.4 
hours  on  pass  2.  The  reported  times  for  pass  1 include  some  general 
problem  familiarization  not  repeated  in  pass  2.  No  statistical  dif- 
ferences exist  between  PDL  or  flowchart  conditions  or  between  experimental 
groups  on  either  the  total  time  or  any  of  its  components.  It  should  be 
noted  that  subjective  reports  of  time  expenditures  are  not  generally  a 
very  reliable  or  sensitive  measure.  In  any  event,  these  data  provide  no 
basis  for  concluding  that  either  documentation  technique  requires  more 
time  than  the  other. 


able  2.  Means  and  (In  Parentheses)  Standard 
Deviations  of  Reported  Design  Times  (hours) 
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Quality  and  Properties  of  Designs 

Unlike  an  implemented  program,  which  can,  at  least  conceivably,  be 
evaluated  by  subjecting  it  to  objective  test,  the  quality  of  unimplemented 
software  designs  can  be  assessed  only  by  subjective  evaluation.  Further- 
more, in  this  particular  study,  the  use  of  different  documentation  methods 
precluded  "blind"  ratings.  Clearly,  the  individual  evaluating  the  design 
would  be  aware  of  the  experimental  condition  in  which  the  design  was  gen- 
erated. Mapping  all  the  designs  into  a common  notation  was  considered, 
but  was  not  feasible  without  at  least  reducing  the  "naturalness"  of  the 
transformed  designs. 

Within  these  constraints,  quality  ratings  were  obtained  quite 
carefully,  with  considerable  attention  by  the  rater  (JRV)  to  the  possi- 
bility of  experimenter  bias.  A priori  grading  criteria  were  developed 
(Figure  2 ) and  applied  to  the  designs.  The  resulting  design  quality 
scores  are  summarized  in  Table  3.  Because  the  pass  1 and  pass  2 
assembler  designs  were  dissimilar  in  function,  the  grading  categories 
do  not  correspond.  The  total  scores,  however,  are  intended  to  be  com- 
parable in  significance,  and  are  on  a scale  from  zero  to  100,  where  a 
higher  score  represents  better  performance. 

Although  it  would  have  been  preferable  to  use  additional  raters, 
both  to  improve  rating  reliability  and  to  provide  a measure  of  that 
reliability,  the  only  other  available  person  with  the  requisite  qualifi- 
cations was  the  course  instructor.  Since  it  was  necessary  to  separate 
the  administration  of  the  experiment  from  the  course  being  taught,  the 
instructor  could  not  be  used  as  a rater.  Since  this  task  requires  a 
large  amount  of  time  and  resources,  additional  raters  were  not  sought. 

Figure  3 provides  a graphical  illustration  of  the  effect  of  the 
experimental  conditions  on  total  design  score.  A corresponding  analysis 
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Pass  1 grading  categories 
100  point  total 

1)  20  points 

Location  counter  logic 

2)  30  points 
Error  detection 

Invalid  operation  codes 
Labels  - req..  ired 
Multiple  label  definitions 
Out  of  range  location  counter 
Premature  end  of  file 

3)  20  points 

Symbol  table  entry  for  defined  symbols 

4)  15  points 

Output  for  input  to  pass  2 

5)  15  points 
General  logic 

Pass  2 grading  categories 
100  point  total 

1)  20  points 
Error  detection 

Undefined  Symbols 
Out  of  range  absolute  addresses 
and  constants 
* Premature  end  of  file 

2)  25  points 

Object  word  creation 

3)  40  points 

Text  buffer  management 
Initialization 

Conditions  for  buffer  output 
Logic  for  building  buffer 
Trailer  record 

4)  15  points 
General  logic 


Note:  "General  logic"  categories  are  intended  for  factors  which  do  not  fall 
under  the  other  specified  categories. 


Figure  2.  Grading  Criteria  for  Assembler  Designs 


Table  3.  Means  and  (standard  Deviations) 
of  Design  Quality  Ratings 


(4.57)  (3.83)  (5.58)  (1.35)  (11.25 


of  variance  is  shown  in  Table  4.  Rated  performance  is  significantly  lower 
on  pass  2 than  on  pass  1 (p  < .05).  These  design  tasks  are  dissimilar, 
although  related,  and  we  had  no  particular  expectation  with  respect  to 
their  relative  difficulty.  The  grading  categories  and  criteria  are  neces- 
sarily different,  and  any  temporal  effects  are  confounded  with  the  pass  1 - 
pass  2 difference.  As  one  might  expect,  given  the  difference  in  ability 

between  the  groups  (more  precisely,  the  difference  in  GPAs),  the  FL-PDL 
group  performed  somewhat  better,  although  not  significantly  better,  than 
the  PDL-FL  group  (p  < .10). 


i 


The  most  relevant  result  of  this  analysis,  however,  is  the  signifi- 
cant difference  in  performance  as  a function  of  documentation  method 
(p  < .025).  Overall  performance  was  better  when  the  design  was  specified 
in  PDL  form  than  when  it  was  specified  in  flowchart  form. 

It  should  be  noted  that  for  Latin  square  designs,  such  as  that  used 
here,  it  is  assumed  that  all  interactions  will  be  zero.  This  is  a neces- 
sary assumption,  since  main  effects  are  confounded  with  interactions,  and 
there  are  not  enough  degrees  of  freedom  to  statistically  separate  the 
interaction  effects.  For  example,  in  the  present  design,  any  POL-FL  vs. 
FL-PDL  group  difference  is  not  separable  from  a documentation-method-by- 
pass  interaction.  Although  such  an  Interaction  may  be  theoretically  very 
interesting,  the  observed  significant  difference  between  the  GPAs  for 
these  two  groups  leads  us  to  conclude  that  this  is,  in  fact,  a group 
difference  and  not  an  interaction. 

In  either  case,  however,  whether  this  is  Interpreted  as  a group 
difference  or  an  interaction.  Its  presence  reduces  the  probability  of 
observing  significant  effects  on  the  other  variables,  including  documen- 
tation method.  That  is,  this  results  in  a conservative  analysis  and  the 
actual  significance  levels  obtained  for  documentation  method  are  somewhat 
lower  than  those  reported,  although  it  cannot  be  determined  how  much 
lower  (cf.  McNemar,  1969,  pp.  383-387). 
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Table  4.  Analysis  of  Variance  of 
Total  Design  Scores 
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Based  on  the  total  design  scores,  a single  design  was  selected 
from  each  of  the  four  groups  for  inclusion  in  this  report.  These  designs 
constitute  Appendix  P,  and  represent  the  most  nearly  "average"  design 
(in  terms  of  design  score  only)  from  each  condition. 

The  designs  were  analyzed  for  differences  in  style,  level  of  detail, 
and  types  of  constructs  used  which  might  be  related  to  documentation 
method.  Factors  analyzed  included  the  number  of  modules  used  in  the 
design,  the  number  of  Boolean  expressions  (conditional  tests),  sub- 
routine call  operations,  and  other  executable  operations  used,  and  the 
extent  and  type  of  abbreviation  used  in  the  design.  The  results,  summar- 
ized in  Table  5,  are  discussed  in  the  following  paragraphs. 

PDL  and  flowchart  conditions  differed  somewhat  with  respect  to  the 
number  of  modules  defined  in  the  designs.  Subjects  tended  to  use  more 
modules  in  designs  produced  in  PDL  form  than  in  designs  produced  in  flow- 
chart form  (F(l,18)  = 3.436,  p < .10). 

It  seems  possible  with  different  documentation  methods  that  de- 
signers might  vary  the  kinds  of  information  contained  in  the  design  and 
the  constructs  used  to  express  it.  Such  differences  are  very  difficult 
to  detect  in  an  objective  manner.  Several  manual  scoring  algorithms  were 
developed  and  rejected  for  measuring  usage  of  particular  design  con- 
structs (number  of  conditional  expressions,  transfers  of  control,  call 
statements,  etc.)  before  adopting  one  which  seemed  meaningful,  unbiased, 
and  relatively  unambiguous  when  applied  to  either  design  documentation 
type.  By  way  of  illustrating  the  problems  encountered,  consider  transfer 
of  control  operations.  In  the  context  of  either  documentation  type  con- 
sidered in  isolation,  it  is  relatively  easy  to  count  transfer  of  control 
operations.  In  a PDL,  one  might  count  GOTO  and  RETURN  statements,  while 
in  flowcharts  one  might  count  RETURN  operations  and  arcs  emanating  from 
decision  diamonds.  The  difficulty  is  that  these  measures  are  not  equiva- 
lent, across  documentation  methods,  in  any  meaningful  sense.  Many 


transfer-of-control  operations  which  are  represented  explicitly  in  a 
flowchart  may  be  represented  implicitly  in  a POL  (as  in  an  IF... THEN 
statement,  DO  WHILE,  procedure  block  with  an  END  but  no  RETURN,  etc.). 
There  are  other  complicating  considerations  too  detailed  to  warrant 
discussion  here. 

The  approach  actually  employed  was  to  count  the  conditional 
(Boolean)  expressions,  call  operations,  and  other  executable  operations 
in  each  design.  Conditional  expressions  are  easily  recognized  in  deci- 
sion diamonds  and  DO  WHILE  blocks  in  flowcharts  and  in  IF... THEN,  DO 
WHILE,  and  similar  constructs  in  a PDL.  The  number  of  conditional 
expressions  is  probably  related  to  the  number  of  transfers  of  control 
required  to  implement  the  design,  although  it  may  clearly  be  affected  by 
the  level  of  detail  of  the  design.  The  number  of  subroutine  call  opera- 
tions might  be  related  to  modular  structure,  but  may  also  be  affected  by 
the  provision  of  an  explicit  called-subroutine  convention  in  the  flow- 
chart method  used.  The  three  categories,  taken  together,  include  all 
the  executable  operations  of  each  design,  but  exclude  such  purely 
structural  statements  as  noniterative  DO,  END  statements,  etc.  The  sum 
of  the  three  counts  is  the  best  metric  we  were  able  to  devise  for 
measuring  the  level  of  detail  of  the  designs. 

Table  5 contains  the  results  of  this  analysis.  The  principal 
difference  observed  was  a highly  significant  tendency  for  PDL  designs 
to  contain  more  conditional  expressions  than  flowchart  designs 
(F(l,18)  = 11.44,  p < .005).  Although  the  PDL-FL  and  FL-PDL  groups 
differed  in  their  frequency  of  use  of  call  operations  (F(l,18)  * 7.21, 
p < .05)  and  other  executable  operations  (F(1.18)  = 4.54,  p < .05), 
frequency  of  use  of  these  constructs  was  not  related  to  documentation 
type.  PDL-FL  and  FL-PDL  groups  also  differed  with  respect  to  total 
number  of  constructs  used  (F( 1 , 18)  = 6.28,  p < .05),  but  on  that  measure 
there  was  also  a documentation  type  difference  (F( 1 , 18 ) = 4.30,  p < .06). 


Pass  1 


Pass  2 


Factor 


Number  of  modules 


Constructs  used 


Call  operations 


Other  executable 
operations 


Total 


Number  of  variable 
names  abbreviated 


9.50 

3.60 

8.20 

3.00 

(7.62) 

(3.41) 

(6.30) 

(2.67) 

37.4 

31.7 

40.5 

28.2 

(8.04) 

(12.82) 

(13.97) 

(9.70) 

62.7 

51.8 

67.7 

42.1 

(17.9) 

(18.3) 

(20.0) 

(16.1) 

7.70 

2.90 

6.40 

4.80 

(6.00) 

(3.11) 

(4.86) 

(3.85) 

t 


i 
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The  latter  difference,  which  suggests  a trend  toward  more  detail  in  the 
PDL  designs  than  in  the  FL  designs,  is  partially,  but  not  entirely, 
attributable  to  the  difference  already  noted  in  usage  of  conditional 
expressions. 


Detailed  study  of  the  designs  produced  with  the  two  documentation 
methods  leads  to  a somewhat  clearer  picture,  and  suggests  that  the  dif- 
ferences in  design  quality  scores  and  level  of  detail  are  related. 

Designs  expressed  in  flowchart  form  tended  to  contain  greater  detail 
in  such  areas  as  initialization  than  those  expressed  in  a PDL,  but  much 
less  algorithmic  detail.  For  example,  a flowchart  design  might  specify 
"X=0",  "Y=0" , "Z*l",  while  a PDL  design  might  merely  say  "INITIALIZE  X, 

Y ,Z" . The  flowchart  design  might  only  say  "COMPUTE  LOCATION  COUNTER 
ADDRESS",  though,  while  the  PDL  design  describes  the  actual  procedure 
for  this  computation.  Somehow,  these  two  documentation  media  seem  to 
lead  to  a different  emphasis  on  the  part  of  the  designer.  It  is  important 
to  note  that  this  observation  holds  true  even  though  the  same  subjects 
used  both  methods,  and  is  not  a result  of  the  group  difference. 

Designs  expressed  in  flowchart  form  exhibited  a marked  tendency 
toward  the  use  of  abbreviated  variable  names  and  other  space-conserving 
practices.  This  tendency  is  probably  related  to  the  need  to  compress 
a lot  of  information  into  the  rather  confined  space  available  in  flow- 
chart blocks,  and  was  observed  to  a much  lesser  degree  in  PDL  designs. 

In  pass  1,  flowchart  subjects  abbreviated  an  average  of  7.7  unique  variable 
names  in  their  designs,  while  PDL  designs  contained  an  average  of  only  2.9 
abbreviated  symbols.  This  difference  is  significant  (F( 15 ) = 2.246,  p < .05). 
Data  from  pass  2 are  ambiguous,  since  subjects  tended  to  continue  usage  of 
the  abbreviations  developed  in  pass  1,  even  though  they  switched  documen- 
tation methods. 

It  is  informative  to  consider  also  the  way  in  which  subjects 
abbreviated  variable  names.  Table  6 lists  the  abbreviations  used,  and 

the  number  of  subjects,  by  documentation  type,  whose  designs  contained 

/ 
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these  abbreviations.  There  appears  to  be  a tendency  for  flowchart  designs 
to  contain  a larger  proportion  of  abbreviated  variable  names  whose  signi- 
ficance is  not  clear  from  the  names  themselves  (i.e. , "non-mnemonic" 
variable  names  like  HL,  II,  etc.). 

Another  observation  by  the  rater  was  that  the  selection  of  appro- 
priate data  structures  by  the  designers  was  very  strongly  related  to 
their  overall  design  scores.  The  rater  divided  the  subjects  into  two 
groups  (best  half,  wdrst  half)  on  the  basis  of  data  structure  usage. 

The  average  overall  design  scores  (both  passes)  for  these  groups,  155.6 
and  104.0,  respectively,  differ  significantly  (t(18)  * 4.24,  p < .001). 
Clearly,  this  is  not  an  a priori  hypothesis,  nor  are  these  two  subjective 
assessments  (ratings  of  data  structure  usage,  overall  design  quality) 
in  any  sense  independent.  Even  if  they  were,  it  would  not  be  clear  to 
what  degree  data  structure  usage  is  a causative  factor  in  design  performance 
and  to  what  degree  both  are  just  similarly  affected  by  subject  differences 
in  ability,  background,  etc.  The  observation  is  interesting,  though, 
and  suggests  an  area  for  further  investigation. 

SUBJECTIVE  RATINGS  OF  DOCUMENTATION  METHODS 


A statistical  analysis  of  the  Flowchart-PDL  comparative  question- 
naire (Appendix  J)  revealed  several  significant  preferences  or  opinions, 
by  the  subjects,  with  respect  to  the  two  documentation  methods.  Each 
response  to  items  1-27  and  29  was  encoded  on  a scale  of  1-8,  where  a 
score  of  1 represents  an  extreme  preference  for  flowcharts  and  8 represents 
an  extreme  preference  for  PDLs.  Since  the  ratings  were  obtained  after 
all  subjects  had  been  exposed  to  the  use  of  both  documentation  methods, 
the  PDL-FL  group  and  the  FL-PDL  group  were  not  expected  to  differ  sig- 
nificantly in  their  responses  to  the  questionnaire.  This  was  verified 
by  a series  of  t-tests,  comparing  the  responses  of  the  two  groups  on  each 
item.  No  overall  pattern  of  differences  emerged,  and  the  number  of 


significant  group  differences  was  at  the  chance  level.  This  allowed 
us  to  combine  the  groups  for  the  purpose  of  analyzing  overall  pre- 
ferences. Means  and  standard  deviations  of  the  ratings  by  all  20  subjects 
are  shown  in  Table  7. 

The  questionnaire  items  were  included  because  they  might 
generate  some  insight  into  designer  and  programmer  perceptions  of  the 
documentation  methods.  In  most  cases,  no  clear  basis  existed  for  an 
a priori  hypothesis  concerning  the  direction,  or  even  the  occurrence, 
of  a preference.  In  eight  cases,  though,  an  a priori  hypothesis  was 
made,  and  the  direction  of  expected  preference  is  indicated  in  paren- 
theses in  the  corresponding  row  of  Table  7. 


Statistical  analyses  indicated  that  four  of  these  items  showed 
at  least  a moderate  (p  < .10)  deviation  from  4.50,  the  value  expected  if 
subjects  have  no  preference  or  opinion.  These  four  items  are  among  the 
eight  on  which  a priori  hypotheses  were  stated,  and  are  in  the  expected 
direction.  Subjects  felt  that  PDLs  were  preferable  to  flowcharts  with 
respect  to  ease  of  implementation  in  a programming  language  ( t ( 19 ) = 2.76, 
p < .02),  ease  of  documentation  ( t( 19 ) = 2.68,  p < .02),  encouragement  of 
"good"  design  practices  (t( 19)  = 2.45,  p < .05),  and  ease  of  modification 
of  documentation  ( t( 19 ) = 1.77,  p < .10).  On  overall  preference  measures, 
the  subjects  exhibited  mild  (non-significant)  preferences  for  PDLs  for 
use  by  the  designer,  but  indicated  an  expectation  that  flowcharts  are 
preferable  from  the  viewpoint  of  the  programmer.  It  should  be  noted  that 
this  questionnaire  administration  preceded  the  programming  task.  This  is 
the  only  item  (of  eight)  on  which  the  direction  of  preference  was  opposite 
that  predicted  a priori. 


i 
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Questionnaire  items  16-27  asked  how  PDLs  and  flowcharts  compare 
with  respect  to  helping  the  designer  notice  problems  with  the  design. 
Question  28  was  intended  to  determine  whether  the  students  understood 
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the  design  properties  referred  to  in  those  questions.  Seven  subjects 
reported  that  questions  16-27  were  easy  to  answer,  eight  students  in- 
dicated that  they  were  difficult  to  answer  because  the  subjects  saw  no 
difference  between  the  documentation  methods,  and  five  indicated  diffi- 
culty understanding  the  design  errors  and  problems  referred  to  in  the 
questions. 


Table  7.  Means  and  (in  Parentheses)  Standard  Deviations 
of  Questionnaire  Ratings  after  Design  Phase 


.Item  Mean  (Standard  Deviation) 


1. 

Ease  of  use  by  designer  (PDL)** 

5.00  (1.95) 

2. 

Ease  of  use  by  programmer  (PDL) 

4.15  (1.79) 

3. 

Helping  designer  understand  problem 

► 

4.35  (1.79) 

4. 

Encouragement  of  "good"  design  practices  (PDL) 

5 . 40* (1.64) 

5. 

Ease  of  documentation  (PDL) 

5.50*(1.67) 

6. 

Ease  of  modification  of  documentation 

(PDL) 

5.05*(1.39) 

7. 

Ease  of  modification  of  design 

4.00  (1.70) 

8. 

Expression  of  control  flow  (FL) 

3.80  (2.24) 

9. 

Expression  of  data  flow 

4.15  (1.87) 

10. 

Comprehensibility  of  design 

4.95  (1.93) 

11. 

Ease  of  reading 

4.65  (1.95) 

12. 

Understanding  pf  control  flow 

4.35  (1.98) 

13. 

Understanding  of  data  flow 

4.40  (1.79) 

14. 

15. 

Ease  of  implementation  in  programming 
Detection  of  design  problems 

language 

(PDL) 

6.00*0.62) 
4.30  (1.84) 

16. 

Detection  of  redundant  operations 

4.25  (1.86) 

17. 

Noticing  that  module  is  too  big 

4.95  (1.50) 

18. 

Noticing  that  module  is  too  complex 

4.45  (1.67) 

19. 

Module  has  too  many  or  too  few  submodules 

4.55  (1.54) 

20. 

Module  is  not  cohesive 

5.03  (1.44) 

21. 

Definition  is  not  detailed  enough 

4.35  (1.57) 

22. 

Definition  has  too  much  detail 

4.35  (1.53) 

23. 

Detection  of  incomplete  module 

4.30  (1.75) 

24. 

Modules  tightly  coupled 

4.55  (1.70) 

25. 

Design  too  complex 

4.05  (1.67) 

26. 

Data  Fragmentation 

4.30  (1.22) 

27. 

Module  is  trivial 

4.80  (1.70) 

29. 

Overall  impression  (pql) 

4.55  (1.64) 

* Significantly  different  from  chance  (4.5).  See  text. 

**  Indicates  an  a priori  hypothesis  that  PDL  would  be  preferred. 


IMPLEMENTATION  PHASE 

In  the  implementation  phase,  each  group  was  exposed  to  only  one 
documentation  method.  If  the  two  groups  of  subjects  had  been  found  to 
differ  with  respect  to  relevant  ability  or  background  variables,  as 
occurred  in  the  design  phase,  that  difference  would  have  been  confounded 
with  the  effect  of  documentation  type.  However,  the  two  implementation- 
phase  experimental  groups  (FL,  PDL)  did  not  differ  significantly  on  any 
known  background  or  ability  factors.  Any  performance  differences  in  the 
implementation  phase  may  therefore  be  attributed  to  the  effects  of  docu- 
mentation  type. 

I M 

The  subjects  from  each  design  phase  group  were  distributed  equally 
into  FL  and  PDL  implementation  phase  groups.  Experience  during  the  design 
phase  is  therefore  counterbalanced  in  the  implementation  phase.  Factorial 
analyses  of  several  key  implementation-phase  variables  (e.g.,  scores  on 
design  comprehension  test,  grades  on  implementation  project,  properties 
of  implemented  program)  reveals  no  pattern  of  interaction  between  design 
and  implementation  conditions.  Therefore,  implementation-phase  results 
will  be  reported  only  by  implementation  condition  (FL  or  PDL).  Because  of 
missing  data  and  other  indications  by  subjects  that  they  were  unable  to 
produce  reliable  estimates  of  their  time  expenditure  during  the  implemen- 
tation phase,  these  data  will  not  be  reported. 

Comprehension  of  Simulator  Design 


Observed  effects  of  documentation  type  on  design  comprehension  were 
small  and  not  statistically  significant.  Table  8 shows  the  results  of  the 
multiple-choice  portion  of  the  design  comprehension  test  administered  after 
45  minutes'  study  of  the  design  (first  test)  and  again  at  the  end  of  the 
entire  implementation  phase  (second  test).  It  is  important  to  note  that 
results  reported  for  the  first  test  are  based  on  all  20  subjects,  while 
those  for  the  second  test  exclude  subjects  who  did  not  complete  the  study. 
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First  Test 


Second  Test 


Category 

Items 

FL 

PDL 

FL 

PDL 

Total  Score 

All 

17.0 

(3.86) 

17.1 

(5.72) 

21.9 

(3.48) 

20.6 

(4.90) 

Comprehension  of  Micro 

Nova 

5,  14,  26 

1.80 

(.789) 

m 

2.75 

(.463) 

2.75 

(.463) 

Distinction  between  Micro- 
Nova  and  Simulator 

11,  12,  24 

■ 

1.80 

(.919) 

2.00 

(.535) 

1.63 

(.916) 

Debugging  of  Simulator 

2,  10,  20, 

29 

1.80 

(1.03) 

2.00 

(1.25) 

2.88 

(.641) 

BBIj 

Comprehension  of  Impact 
of  Changes 

6,  9,  19, 

27 

1.50 

(1.18) 

2.b3 

(1.06) 

3.13 

(.991) 

Comprehension  of  Gen- 
eral Structure 

4,  8,  15, 

16,  21,  23, 

25,  28 

4.10 

(.876) 

4.60 

(1.90) 

4.75 

(1.04) 

5.00 

(1.69) 

Comprehension  of  Pro- 
cedural Detail 

1,  3,  7,  13, 
17,  18,  22,  30 

5.20 

(2.26) 

4.70 

(2.26) 

6.50 

(1.77) 

6.00 

(1.60) 

On  the  total  score,  and  most  of  the  subscores,  statistically  significant 
improvement  was  observed  from  the  first  test  to  the  second,  as  might  be 
expected. 

On  the  first  test,  students  using  a PDL  scored  somewhat  better  than 
FL  subjects  on  comprehension  of  the  MicroNova  (t( 18)  = 2.090,  p < .06); 
this  difference  disappeared  entirely  on  the  second  test.  It  should  be 
noted  that  this  test  category  (as  well  as  the  next  category.  Distinction 
between  MicroNova  and  Simulator)  was  intended  as  a measure  of  background 
knowledge,  rather  than  of  information  contained  in  the  design  itself.  It 
is  possible,  however,  that  the  PDL  design  somehow  aided  rehearsal  of 
this  information,  or  otherwise  reinforced  it  more  than  did  the  flowchart 
design.  The  MicroNova  information  is  derivable  from  the  design;  it  was 
considered  background  information  primarily  because  the  student  was  ex- 
pected to  know  it  already. 

Repeated-measures  analyses  of  variance  on  the  total  score  and  each 
subscore  (using  data  for.  the  16  subjects  who  completed  the  study)  found  no 
flowchart-PDL  differences  which  were  consistent  across  both  test  adminis- 
trations. 

As  a priori  hypothesis  relating  to  subscores  was  that  flowcharts 
might  aid  comprehension  of  procedural  detail,  while  PDLs  might  improve 
comprehension  of  the  general  structure  and  function  of  the  design.  The 
data  indicate  mild,  but  statistically  insignificant,  tendencies  in  the 
expected  directions. 

No  significant  differences  were  observed  between  PDL  and  FL  subjects 
with  respect  to  the  number  of  modules  they  were  able  to  correctly  identify 
from  the  design,  or  with  respect  to  the  number  of  call ing-module/cal led- 
module  relationships  correctly  identified.  These  are  probably  not 
sensitive  measures  of  design  comprehension. 
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Quality  of  Implementations 

We  originally  intended  to  evaluate  the  simulator  programs  by  the 
method  of  Youngs  (1974),  in  which  the  final  and  all  earlier  runs  are 
analyzed  for  errors  in  reverse  chronological  order.  This  method  is  very 
effective  in  detecting  a variety  of  error  types,  by  the  programmer,  in- 
cluding clerical,  snytactic,  and  various  types  of  conceptual  error. 
However,  the  number  of  runs  involved  was  much  larger  than  anticipated 
(1192),  and  such  an  approach  would  have  been  an  overwhelming  task,  par- 
ticularly given  the  size  of  the  programs  (average  length,  1030  source 
statements).  As  a result,  only  the  final,  completed  programs  were 
evaluated. 

The  completed  simulator  programs  were  evaluated  by  an  expert  rater 
(JRV)  on  the  basis  of  the  a priori  grading  criteria  shown  in  Figure  4. 
These  ratings  were  done  on  the  basis  of:  (1)  the  finished  source  programs 
(as  compiled,  with  symbol  table,  cross-reference  table,  etc.),  (2)  results 
of  a fairly  extensive  test  program,  and  (3)  reference  to  earlier  simulator 
development  runs  as  needed.  The  results  of  this  evaluation  are  shown  in 
Table  9.  Statistical  analysis  indicates  that  there  are  no  significant 
differences  between  flowchart  and  POL  groups  on  total  score  or  on  any  of 
the  subscores.  No  particular  classes  of  implementation  error  were  noted 
which  seemed  to  be  attributable  to  design  documentation,  type. 

/ 

| 

Appendix  Q contains  the  program  which  was  most  nearly  average  in 
quality.  This  particular  implementation  was  developed  from  a flowchart 
design. 

The  final  programs  were  also  evaluated  with  respect  to  several 
simple  measures  of  programming  style  or  programming  practice.  No  dif- 
ferences were  observed  between  FL  and  PDL  subjects  with  respect  to 
number  of  unique  variable  names,  total  number  of  variable  name  references. 
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Implementation  Evaluation  Categories 
Category  1 (50  points) 


Location  counter  modification  and  referencing  logic 
Incrementation  associated  with  fetching 
Conditional  modification  based  on  skip  indicator 
Relative  addressing 
Jumps  and  subroutine  calls 
Traps  and  interrupts 
Subroutine  returns 
Two  word  block  I/O  instructions 
Setting  and  testing  active  trace  location 


Category  2 (50  points) 


Effective  address  computation  and  usage 

Indexing  and  displacement  addressing 
Indirect  addressing  including  chaining 
and  auto  indexing 
Usage  of  effective  addresses 
Interrupts  and  traps 
I/O 

Memory  reference  instructions 


Category  3 (15  points) 

Dump  and  trace  control  logic 
Category  4 (25  points) 


Interrupt  detection  and  generation  including  real  time  clock 
management  and  interrupt  enable/disable  logic. 


Category  5 (15  points) 

Instruction  class  resolution  and  invocation. 

Category  6 (25  points) 

Instruction  simulation  excluding  logic  covered  by  previous  categories 
Total  - 180  points 


Figure  4.  Grading  Criteria  for  Implemented  Simulators 
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or  number  of  usages  of  each  variable.  The  flowchart  group  used  26% 
more  "60  TO"  statements  than  did  the  POL  group  (means  were  36.75  and 
29.13,  respectively),  and  this  difference  remained  when  adjustments  were 
made  from  program  length.  However,  the  variances  were  very  large,  and 
this  mean  difference  does  not  approach  statistical  significance. 

The  FL  and  PDL  groups  also  exhibited  differences  in  average  program 
length  and  in  the  number  of  runs  made  during  program  development.  As  in 
the  case  of  "GO  TO"  statements,  the  mean  differences  are  fairly  large, 
but  are  not  statistically  significant.  The  FL  and  PDL  subjects  averaged 
63  and  86  runs,  respectively,  but  this  difference  is  not  significant 
(F  (1,  14)  = 2.272,  p > .10).  The  average  program  length  (number  of 
source  statements)  for  FL  and  PDL  subjects,  respectively,  was  1136  and 
924,  a non-significant  difference  (F  (1,  14)  = 1.893,  p > .10). 

Subjective  Ratings  of  Documentation  Methods 


Results  of  the  post-implementation  comparative  questionnaire  are 
shown  in  Table  10.  Overall,  subjects  indicated  significant  preference 
for  PDL  over  flowchart  documentation  (t  (15)  = 2.15,  p < .05).  Specifi- 
cally, significant  PDL  preferences  were  noted  with  respect  to  ease  of 
documentation  (t  (15)  = 4.04,  p < .01),  ease  of  modification  of  documen- 
tation (t  (14)  = 2.40,  p < .05),  ease  of  implementation  in  a programming 
language  (t  (15)  = 3.95,  p < .01),  and  ability  to  detect  an  insufficiently 
detailed  module  in  the  design  (t  (15)  = 2.18,  p < .05).  Students  tended 
also  to  prefer  PDLs  with  respect  to  encouragement  of  "good"  design 
practices  (t  (15)  = 2.06,  p < .10).  On  questionnaire  items  involving 
a priori  hypotheses,  eight  out  of  eight  preferences  were  in  the  hypothe- 
sized direction,  with  four  significant  at  the  P < .05  level  (and  one  trend 
at  P < .10). 

The  subjective  ratings  may  have  been  biased  by  a lecture  delivered 
at  0SU  by  well-known  computer  scientist  Daniel  McCracken  shortly  before 


i 
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the  questionnaire  was  administered.  In  the  lecture,  McCracken  extolled 
the  virtues  of  PDLs  at  some  length.  With  respect  to  overall  preference 
(item  29),  subjects  attending  the  lecture  produced  a mean  rating  of  6.25, 
as  compared  with  4.75  for  those  not  attending  the  lecture.  Variances 
were  large,  however  (standard  deviations  were  1.28  and  2.12,  respectively), 
and  this  mean  difference  does  not  achieve  statistical  significance 
(t  (13)  * 1.71,  p > .10).  The  cause  and  effect  are  not  entirely  clear, 
as  the  more  highly  motivated  students  were  probably  more  prone  to  attend 
the  lecture.  Nonetheless,  this  biasing  factor  must  be  considered  in 
interpreting  the  results. 

The  results  shown  in  Table  10  actually  represent  two  groups  (POL, 

FL)  which  were  not  entirely  in  agreement.  As  one  might  expect,  subjects' 
preferences  tended  to  be  influenced  by  the  documentation  method  they 
actually  used  during  the  implementation  phase.  On  most  questionnaire 
items,  PDL  subjects  indicated  stronger  preference  for  PDLs  than  did  FL 
subjects.  These  group  differences  were  only  significant  on  a few  items, 
however,  on  item  12,  understanding  of  control  flow,  mean  ratings  were  5.75 
and  3.63,  respectively  (t( 14)  = 2.26,  p < .05).  On  item  15,  detection  of 
design  problems,  means  were  5.88  and  4.13  (t  (14)  = 3.30,  p < .01). 

Immediately  after  the  initial  study  of  the  designs  and  administra- 
tion of  the  first  comprehension  test,  several  students  using  flowcharts 
commented  spontaneously  that  their  impression  of  the  relative  readability 
of  the  two  documentation  methods  had  changed  considerably  as  a result  of 
their  attempt  to  understand  the  flowchart  design.  This  was  reflected 
later  in  their  responses  on  item  11  of  the  questionnaire.  Between  admin- 
istrations of  the  questionnaire,  the  ratings  on  this  item,  by  students 
using  flowcharts,  changed  by  an  average  of  1.38  points  in  favor  of  PDLs 
(by  a difference  test,  (t  (7)  = 4.88,  p < .01).  The  responses  of  both 
POL  and  FL  subjects  also  changed  an  average  of  .94  points  in  favor  of 
POLs  on  item  21,  detection  of  insufficiently  detailed  module  (t  (15)  = 4.04, 


Table  10.  Means  and  (Standard  Deviations)  of  Questionnaire 
Ratings  after  Implementation  Phase 


Item  Mean  (Standard  Deviation) 


1.  Ease  of  use  by  designer  (PDL)** 

5.25  (1.77) 

2.  Ease  of  use  by  programmer  (PDL) 

5.13  (2.06) 

3.  Helping  designer  understand  problem 

5.0  (1.79) 

4.  Encouragement  of  "good"  design  practices  (PDL) 

5. 56*(2.06) 

5.  Ease  of  documentation  (PDL) 

5 . 88* (1.36) 

6.  Ease  of  modification  of  documentation  (PDL) 

5.4  *(1.45) 

7.  Ease  of  modification  of  design 

4.73  (1.87) 

8.  Expression  of  control  flow  (FL) 

3.80  (1.86) 

9.  Expression  of  data  flow 

4.0  (1.52) 

10.  Comprehensibility  of  design 

5.0  (1.96) 

11.  Ease  of  reading 

5.0  (1.78) 

12.  Understanding  of  control  flow 

4.69  (2.12) 

13.  Understanding  of  data  flow 

4.47  (1.46) 

14.  Ease  of  implementation  in  programming  language 

6.2  5* ( 1 .77) 

15.  Detection  of  design  problems  (p0L) 

5.0  (1.37) 

16.  Detection  of  redundant  operations 

4.38  (1.45) 

17.  Noticing  that  module  is  too  big 

4.75  (1.44) 

18.  Noticing  that  module  is  too  complex 

4.38  (1.50) 

19.  Module  has  too  many  or  too  few  submodules 

4.81  (1.47) 

20.  Module  is  not  cohesive 

4.63  (1.26) 

21.  Definition  is  not  detailed  enough 

5.31*(1.49) 

22.  Definition  has  too  much  detail 

4.81  (1.42) 

23.  Detection  of  incomplete  module 

4.63  (1.41) 

24.  Modules  tightly  coupled 

4.33  (1.40) 

25.  Design  too  complex 

4.25  (1.53) 

26.  Data  Fragmentation 

4.20  (1.21) 

27.  Module  is  trivial 

4.81  (1.38) 

29.  Overall  impression  (POL) 

5.50*(1 .86) 

* 


★★ 


Significantly  different  from  chance  (4.5).  See  text. 
Indicates  an  a priori  hypothesis  that  POL  would  be  preferred 
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DISCUSSION 

The  results,  indicate  that  designs  produced  in  PDL  form  had 
higher  overall  quality  than  those  produced  in  flowchart  form.  PDL 
designs  were  more  detailed,  particularly  with  respect  to  algorithmic 
detail,  and  involved  much  less  abbreviation  of  variable  names  than 
flowchart  designs.  When  designs  were  presented  to  subjects  in  PDL  or 
flowchart  form,  however,  no  pattern  of  short  or  long-term  differences 
was  detected  on  measures  of  design  comprehension.  No  significant 
differences  were  detected  in  the  quality  or  other  properties  of  the 
programs  written  as  implementations  of  the  designs.  Subjective  ratings 
indicated  a mild  overall  preference  for  PDLs,  as  well  as  several  speci- 
fic criteria  on  which  students  found  PDLs  preferable  to  flowcharts. 

The  measures  of  design  quality  which  were  used  to  assess  subject- 
generated designs  can  be  criticized  as  too  subjective.  However,  it  is 
not  clear  that  any  more  objective  approach  is  available,  given  our 
current  level  of  understanding  of  the  software  design  process.  In  any 
event,  the  differences  in  logical  quality  of  the  designs  were  fairly 
large,  favored  PDLs  fairly  consistently,  and  were  generally  consistent 
with  other,  more  objective  observations  concerning  the  level  of  detail 
and  readability  of  the  produced  designs. 

The  observed  difference  in  level  of  detail  between  flowchart  and 
PDL  designs  is  particularly  interesting.  The  fact  that  PDL  designs  con- 
tained 32%  more  conditional  expressions  than  did  flowchart  designs 
suggests  that  they  contain  a greater  degree  of  algorithmic  or  procedural 
detail.  This  may,  in  fact,  be  a significant  factor  in  their  higher 
assessed  quality.  The  finding  that  PDL  designs  contained  44%  more 
modules  than  did  flowchart  designs  is  also  suggestive  of  greater  detail, 
although  it  is  possible  that  PDLs  simply  encourage  modularization. 
Notice,  however,  that  a higher  level  of  procedural  detail  Implies  the 
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existence  of  a larger  number  of  non-conditional  expressions,  as  well  as 
a larger  number  of  conditional  expressions.  Yet,  a 32%  difference  in 
conditional  expressions  was  accompanied  by  only  a 10%  difference  in 
"other  executable  operations".  Examination  of  the  designs  suggests 
that  the  flowchart  designs  contain  more  detailed  information  concerning 
some  aspect  of  the  design  which  is  not  associated  with  condition- 
testing. A particularly  prevalent  difference  concerns  initialization 
of  variables.  While  POL  designs  tend  to  specify  initialization  with  a 
single  ambiguous  statement  ("INITIALIZE  VARIABLES",  or  something  similar), 
flowchart  designs  tend  to  contain  explicit,  unambiguous  specifications 
of  each  initialization  operation  (e.g.,  "SET  X=0,  "SET  Y=0",  SET  Z=l"). 

Informal  program  design  languages  are  similar  to  natural  language 
to  a much  greater  degree  than  are  flowcharts.  Studies  of  query  formula- 
tion by  non-programmers  (Gould,  et  al  1976;  Miller  and  Becker,  1974)  have 
observed  the  frequent  use  of  procedural ly  ambiguous  statements  which  re- 
quire interpretation  by  the  reader.  The  use  of  such  statements  as 
"INITIALIZE  VARIABLES"  in  PDL  designs  may  be  related  to  those  observa- 
tions. In  using  natural  language,  people  apparently  tend  to  assume 
that  the  recipient  of  the  communication  will  possess  sufficient  semantic 
information  to  correctly  interpret  such  imprecise  statements.  The  flow- 
chart "language",  on  the  other  hand,  is  much  more  formal  and  may  encourage 
the  use  of  detailed,  unambiguous  individual  statements. 

PDL  and  flowchart  usage  differed  not  only  in  overall  level  of 
detail,  but  also  in  the  aspects  of  the  designs  selected  for  detailed 
specification.  Many  would  argue  that  it  is  the  algorithmic  detail  that 
is  most  important  in  the  specification  of  these  designs,  and  that  details 
in  initialization  and  similar  information  can  be  provided  by  the  program- 
mer. If  true,  the  data  suggests  that  the  designer's  use  of  PDLs  encourages 
more  detailed  specification  of  relevant  aspects  of  the  design  than  does 
his  use  of  flowcharts. 


When  subjects  were  presented  with  logically  equivalent  designs 
expressed  either  in  PDl  or  flowchart  form,  no  differences  were  observed 
in  either  "short-term"  (45  minutes)  or  long-term  comprehension  of  the 
designs.  The  lack  of  a short-term  comprehension  difference  seems  incon- 
sistent with  the  findings  of  Wright  and  Reid  (1973),  whose  subjects 
remembered  a procedure  better  when  expressed  in  a form  somewhat  similar 
to  a PDL  than  when  expressed  in  flowchart-like  form.  It  should  be 
noted,  however,  that  comprehension  of  this  simulator  design  is  quite  a 
formidable  task  when  compared  with  their  simple  algorithms,  which 
typically  involved  only  three  binary  decisions.  It  is  questionable 
whether  45  minutes'  exposure  to  the  simulator  design  represent  a "short- 
term" presentation  in  the  same  sense  as  the  much  shorter  exposure  to 
simple  algorithms  in  the  Wright  and  Reid  study.  Furthermore,  their 
recall  measure  (execution  of  algorithm  from  memory)  is  probably  a more 
sensitive  performance  measure  than  our  design  comprehension  test.  Finally, 
the  specialized  background  of  our  programmer-subjects  may  have  led  them 
to  develop  more  efficient  methods  for  processing  flowchart  information 
and  encoding  it  in  memory. 

This  experiment  is  a conservative  test  of  the  relative  comprehen- 
sibility of  the  two  documentation  methods.  In  order  to  investigate  dif- 
ferences in  documentation  method  in  isolation  from  other  factors,  it  was 
necessary  to  carefully  control  the  way  in  which  design  statements  were 
worded.  Wherever  possible,  complete  statements  were  expressed  identically 
in  PDL  and  flowchart  designs.  This  resulted  in  flowcharts  which  were 
more  "wordy"  than  usual,  but  which  were  locally  (i.e.,  at  the  level  of 
individual  statements)  quite  readable.  The  primary  issue  addressed  in 
the  experiment  was  the  effect  of  documentation  type  on  global  compre- 
hensibility. 

The  results  suggest  that  flowcharts  and  PDLs  are  equally 
comprehensible  if  they  contain  similar,  readable,  unabbreviated  statements. 
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If  used  well,  either  method  may  allow  effective  communication  with  the 
programmer.  In  actual  practice,  however,  the  use  of  flowcharts  ap- 
pears to  encourage  the  use  of  highly  abbreviated  symbols  and  other 
space-saving  techniques.  Although  the  present  study  has  not  attempted 
to  determine  whether  or  not  such  abbreviations  adversely  affect  the 
comprehensibility  of  designs,  it  would  be  surprising  if  they  did  not  do 
so,  at  least  in  the  short  term.  Thus,  in  actual  practice  flowchart 
designs  may  be  less  comprehensible  because  of  the  medium's  effect  on 
the  behavior  of  the  designer,  rather  than  its  effect  on  the  programmer. 

Design  documentation  method  was  not  found  to  affect  the  quality 


or  style  of  implementation  by  the  programmer.  It  might  be  suggested 
that  a good  programmer  should  be  able  to  understand  a good  design  ex- 
pressed in  any  reasonable  form.  In  fact,  such  a suggestion  was  made  by 
several  of  the  students  who  participated  in  the  study.  The  issue  may 
not  be  that  simple,  however.  The  fact  that  the  flowchart  and  PDL  designs 
were  logically  equivalent  does  not  imply  that  they  were  psychologically 
equivalent.  If  the  programmer's  perception  of  the  design  is  altered  by 
the  form  in  which  the  design  is  presented  --  for  example,  by  causing  him 
to  attend  more  to  control  structure,  or  more  to  modularization  — he  may. 


in  effect,  solve  a different  problem.  There  is  evidence  of  such  phenomena 
in  the  literature  of  other  task  domains  (e.g.,  Simon  & Hayes,  1976).  There 
was  a no  a priori  justification  for  assuming  that  the  peculiar  properties 
of  PDLs  and  flowcharts  would  not  exert  such  differential  effects.  Based 
on  the  data  obtained  in  this  experiment,  however,  such  differential  ef- 
fects appear  to  be  either  insignificant  in  magnitude  or  of  a type  to 
which  the  analyses  employed  are  insensitive.  The  latter  possibility  is 
certainly  not  to  be  ignored. 

Subjective  preferences,  as  expre|sed  in  responses  to  both  question- 
naires, and  especially  the  post-implementation  questionnaires,  indicate  a 
mild  but  somewhat  consistent  preference  for  PDLs  over  flowcharts.  Subjects 
also  indicated  several  criteria  on  which  PDLs  are  particularly  to  be 
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preferred,  including  encouragement  of  "good"  design  practices,  ease  of 
documentation  and  of  modification  of  documentation,  and  ease  of  imple- 
mentation in  a programming  language. 

Overall,  the  results  appear  to  provide  a fairly  strong  case  for 
the  use  of  program  design  languages,  in  preference  to  flowcharts,  for 
the  expression  of  detailed  sottware  designs  by  the  designer.  As  with 
all  studies,  it  must  be  kept  in  mind  that  this  study  was  done  using  a 
particular  design  and  programming  task,  a particular  class  of  designers 
and  programmers,  and  a particular  experimental  setting.  The  degree  to 
which  similar  results  might  be  obtained  in  other  software  design  and 
programming  situations  depends  in  part  upon  their  similarity  to  the 
conditions  under  which  this  experiment  was  conducted.  Generalization  to 
meaningful  software  development  settings  was,  of  course,  the  primary 
consideration  in  our  decision  to  study  the  behavior  of  experienced 
programmers  involved  in  software  development  efforts  of  significant 
size. 


CONCLUSIONS 


y 


In  the  context  in  which  this  study  was  performed,  the  use  of 
a Program  Design  Language  (PDL)  by  a software  designer,  for  the  develop- 
ment and  description  of  a detailed  program  design,  produced  better 
quality  designs  than  did  the  use  of  flowcharts.  In  particular,  the  PDL 
designs  involved  more  algorithmic  or  procedural  detail  than  those  pro- 
duced using  flowcharts.  In  addition,  flowchart  designs  exhibited 
considerably  more  abbreviation  and  other  space-saving  practices  than 
did  PDL  designs,  with  a possible  adverse  effect  on  their  readability. 

When  equivalent,  highly  readable  designs  were  presented  to  subjects 
in  both  PDL  and  flowchart  form,  no  pattern  of  short-term  or  long-term 
differences  in  comprehension  of  the  design  was  observed.  No  significant 
differences  were  detected  in  the  quality  or  other  properties  of  programs 
written  as  implementations  of  the  designs.  Subjective  ratings  indicated 
a mild  preference  for  PDLs. 

Overall,  the  results  suggest  that  software  design  performance  and 
designer-programmer  communication  might  be  significantly  improved  by  the 
adoption  of  informal  Program  Design  Languages,  rather  than  flowcharts. 


as  a standard  documentation  method  for  detailed  computer  program  designs. 
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APPENDIX  B - CONSENT  FORMS 
INFORMED  CONSENT  FORM 


The  purpose  of  this  study  is  to  examine  the  relative  merits  of  two  forms 
of  design  specification,  flowcharts  and  program  design  languages,  on  the 
design  and  implementation  of  software  systems.  This  study  will  in  no  way 
interfere  with  the  stated  objectives  of  this  course.  We  will  observe  and 
record  your  performance  on  required  course  projects;  you  will  not  be  required 
to  perform  significant  activities  in  addition  to  normal  course  assignments. 

We  will  observe  your  performance  on  two  major  assignments  — a design  task 
and  an  Implementation  task.  In  addition,  you  will  be  given  course  exams  and 
asked  to  complete  a questionnaire  describing  your  background  in  computer 
science.  Participants  in  this  study  will  remain  anonymous.  The  sole  purpose 
of  this  research  is  to  examine  alternative  forms  of  design  specification;  it  is 
not  a test  of  your  intelligence  or  personality.  You  are  not  required  to  partici- 
pate in  this  study,  and  you  may  change  your  mind  about  participating  at  any  time 
during  the  experiment.  However,  payment  or  other  rewards  can  only  be  given  to 
those  who  complete  all  experimental  requirements.  At  the  conclusion  of  this 
study,  a complete  description  of  this  research  will  be  given  to  you. 

Although  course  grades  will  be  determined,  in  part,  on  the  basis  of  informa- 
tion collected  during  this  experiment,  experimental  manipulations  will  not  in- 
fluence grades.  Your  performance  will  be  compared,  for  grading  purposes,  only 
with  that  of  students  under  identical  experimental  conditions.  The  experimental 
manipulations  are  not  intended  to  make  your  task  more  difficult  or  to  degrade 
your  performance  in  any  way.  There  is  no  reason  to  believe  that  the  experimental 
materials  that  are  presented  to  you  will  either  help  or  hinder  your  performance 
relative  to  students  given  other  materials.  In  order  to  insure  an  accurate 
assessment  of  the  experimental  manipulations  to  be  performed,  we  ask  that  you 
do  not  compare  materials,  problems  or  solutions  with  fellow  students. 

Any  questions  concerning  the  procedures  of  this  study  can  be  discussed  with 
Dr.  Van  Doren.  These  procedures  were  designed  to  be  consistent  with  guidelines 
established  by  the  Department  of  Health,  Education  and  Welfare  concerning  the 
use  of  human  subjects  and  with  Oklahoma  State  University  policies. 

The  results  of  this  study  are  expected  to  be  of  considerable  importance 
to  the  software  development  community.  Previous  studies  have  generally  in- 
volved undergraduates  with  little,  if  any,  computer  science  background  and  very 
small  projects.  The  results  obtained  from  such  studies  are  frequently  equivo- 
cal. The  study  in  which  you  will  participate  offers  the  significant  opportunity 
to  observe  the  performance  of  well-qualified  computer  scientists  on  non-trivial 
projects.  This  increases  the  probability  of  finding  meaningful  results  that  can 
be  used  to  improve  computer  science  training  and  practices. 

Your  cooperation  is  invaluable  and  greatly  appreciated.  Because  the 
study  is  expected  to  be  interesting  and  beneficial  to  you,  as  well  as  to  the 
software  development  community  at  large,  we  hope  you  will  want  to  participate. 
However,  as  an  additional  reward  for  your  participation,  you  will  be  paid  $25.00 
upon  successful  completion  of  all  experimental  requirements. 
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Or.  H.  Rudy  Ramsey 
SCIENCE  APPLICATIONS,  INC. 

40  Denver  Technological  Center  West 
7935  East  Prentice  Avenue 
Englewood,  Colorado  80110 

I 

f 

I have  read  and  understand  the  above  statement  and  agree  to  participate 
in  this  study. 


In  analyzing  the  results  of  this  study,  it  would  be  useful  to  have  a record 
of  your  previous  and  current  courses  and  the  grades  received.  All  information 
about  grades  will  be  treated  as  confidential  and  identified  only  by  a code  number 
A master  list  assigning  code  numbers  to  names  will  be  maintained  only  by  Dr. 

Van  Doren  and  will  be  destroyed  as  soon  as  possible  after  completion  of  this 
study. 

Subject  to  the  conditions  stated  above,  I agree  to  the  release  of  the  infor- 
mation contained  in  my  academic  record. 


Signature 
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APPENDIX  C - BACKGROUND  QUESTIONNAIRE 


Name: 


INSTRUCTIONS 


This  questionnaire  is  designed  solely  for  experimental  purposes;  your 
responses  will  in  no  way  affect  your  course  grade.  Since  we  will  be 
observing  your  performance  on  course  projects,  it  would  be  very  useful 
to  us  to  have  a brief  description  of  your  background  in  computer  science. 
We  ask  that  you  complete  these  questions  accurately  and  honestly. 


i 

v 

•5 


C-l 


Code: 

1.  List  previous  computer  science  courses. 
Undergraduate 


Graduate 


2.  Are  you  enrolled  in  the  computer  science  program?  Yes No 

If  no,  which  program  are  you  enrolled  in? 

If  yes , which  graduate  option? 

How  many  hours  have  you  completed  in  this  option? 

How  many  graduate  hours  have  you  completed  in  computer  science? 


3.  List  previous  degrees  received 


4.  When  do  you  expect  to  graduate  from  this  program  and  what  degree  will 
you  receive? 


5.  Are  you  a native  speaker  of  English?  Yes No 

If  n£,  how  long  have  you  lived  in  English  speaking  countries? 

Years Months 

Were  your  previous  degrees  obtained  at  universities  where  English 
was  used?  Yes No Some 

If  no  or  some,  briefly  describe  your  educational  background  in  terms 
of  universities,  language,  years  enrolled  and  degrees  received. 


C-2 
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6.  Briefly  describe  your  experiences,  outside  of  normal  course  work  with  the 
following  (list  types  of  experience,  length,  where  received,  etc.): 

PL/1 

Flowcharts 


Machine  Simulators 
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APPENDIX  D - SOFTWARE  DESIGN  METHODS  HANDOUT 

Notes  on  Software  Design  Methods  (Flowcharts  and  PDL's) 


The  material  presented  is  similar  but  not  identical  to  material 
contained  ini 

Structured  Programming  Series,  Vol,  81 
Program  Design  Study 
IBM  Federal  Systems  Center 
Gathersburg,  Maryland 


Produced  under  U.S.  Air  Force  contract 
and  available  from 

National  Technical  Information  Service 
Ref.  # RADC-TR -74-300 


. r.  ..  , \kt-A 


It  is  assumed  that  you  are  familiar  with  .flowcharting  and  the 
conventions  in  use  at  Oklahoma  State  University.  It  is  not  assumed 
that  you  are  familiar  with  ProgramflJef initiorf) Languages  (FDL's), 

PDL's  typically  have  control  structures  simfliar  to  those  available 
in  block  structured  languages  such  as  PL/l  and  ALCOL.  However, 
a PDL  is  used  for  software  design, not  as  a programming  language. 

Of  course  there  may  be  direct  correspondence  with  some  features 
in  your  programming  language  just  as  there  may  be  with  flowcharts. 
PDL's  may  be  informal  or  formal.  The  latter  have  a very  formal 
and  strict  definition.  The  former  have  certain  well  defined  control 
structures  but  much  of  the  processing  can  be  specified  in  an  informal 
way  just -as  flowchart  procesing  blocks  may  have  varying  degrees 
of  precision  and  formality  depending  on  the  level  of  3-ogic  or  design. 

For  purposes  of  the  software  design  study  being  conducted  in 
this  course  an  informal  FDL  will  be  used.  As  part  of  your  computer 
science  knowledge  you  should  become  aware  of  PDL's  as  alternative 
methods  for  software  design  (and  documentation  as  well) . 

The  remainder  of  the  material  comprises  a parallel  presentation 
of  the  control  structures  for  the  informal  PDL  and  corresponding 
flowcharting  treatments . 
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Modules  or  procedures » 

PDL 

Module  name i PROCEDURE;  optional  parameter  list; 


Sequence  of  PDL  and/or 
English  langauge  statements 


RETURN  TO  CALLER; 
END  module  name; 


Flowchart 

Optional j 
^parameter/ 
list 


module  name;  ( ENTRY 


Module  invocation; 

PDL 

CALL  module  name  optional  parameter  list; 


Elementary  decision  logic i 
PDL 

IF  condition  THEH 

English  language  or  PDL  statement; 

ELSE 

English  language  or  PDL  statement; 


Graphical  display 
of  processing  blocks, 
decision  blocks  and 
flow  lines 

[ RETURN  j 


Flowchart 


module  name 


optional 

parameter 

list 


Flowchart 


IF  condition  Then 

English  language  or  PDL  statement; 


Loon in" « 


I 

r 


PDL 

DO  WHILE  condition; 

Sequence  of  English  language 
and/or  PDL  statements; 

END; 


Flowchart 


DO  UNTIL  condition; 

Sequence  of  English  language 
and/or  PDL  statements; 

END* 


DO  index  = initial  value  TO  final  value  BY  incr. 
Sequence  of  English  language 
and/or  PDL  statements; 


END; 


Multiway  decision; 
PDL 


t 


DO  CASE  arithmetic  value;- 
CASEOj 

English  language  statement 
or  PDL  control  statement; 
CASSli 

English  language  statement 
or  PDL  control  statement; 


• 

CAS  Em 

English  language  statement 
or  PDL  control  statement; 
END  CASE; 


Notel:  If  the  arithmetic  value  is  "out  of  range"  nothingis  specified. 
Some  conventions  establish  a default  control  path  for  this.  Please 
observe  that  a zero  arithmetic  value  is  considered  in  range  for  this  PDL. 


Note2i  Multiway  decisions  such  as  illustrated  may  easily  be  simulated 
inPL/l  with  the  use  of  a LABEL  vector,  the  PL/l  equivalent  of  a "computed 
GO  TO"  and  a GO  TO  after  each  CASE  block. 
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Sequence  of  desircn  sneclficiationsi 


i 


PDL 

DO; 

Sequence  of  English  language 
and/or  PDL  statements; 

ENDy 


Flowchart 

1 

• 

% 

% 


Additional  comments; 


Note  that  semicolons  " j”  are  used  to  terminate  all  statments  in 
a PDL  design.  This  may  be  particularly  important  in  making  clear  a 
se<?ence  of  processing  givne  in  the  form  of  English  language  statements. 
Further  note  that  indentation  schemes. for  nested  PDL  specifications 


APPENDIX  E - PDL  AND  FLOWCHART  EXAMPLES 


PDL  AND  FLOWCHART  SOFTWARE  DESIGN  ILLUSTRATIONS 


The  illustrations  are  given  in  terns  of  a simulator  design  for 
the  "small  computer"  discussed  in  class  via  APL  functional  descriptions. 
There  are  several  items  about  this  design  to  be  stressedi 

1)  For  design  illustration  reasons  it  is  assumed  that  an  op  code  of 
0000  (binary  bits)  represents  a "HALT"  instruction  and  that  op  codes 
of  0011,  0100  and  0111  are  invalid.  The  APL  functional  description 
does  not  include  these  items.  It  is  further  assumed  that  op  codes  of 
1100  and  1111  represent  "TRACS  ON"  and  "TRACE  OFF"  simulator  Instructions. 
These  two  instructions  are  not  part  of  the  "small  computer"  instruction 
set  but  rather  are  simulator  design  provisions  for  tracing  the  flow  of 
simulated  execution  of  "small  computer"  programs, 

2)  The  simulator  design  is  a software  design  which  contains  features 
that  are  not  part  of  the  functional  characteristics  of  the  simulated 
machine.  Obviously  there  are  some  similarities  between  the  simulator 
design  and  the  APL  functional  characteristics. 

3)  Neither  of  the  software  designs  includes  the  mathematical  precision 
of  the  the  APL  functional  specifications.  Precise  details  for 
simulating  certain  functional  characteristics  should  be  deduced  from 
the  APL  specifications.  Programming  details  do  not  require  direct 
correspondence  to  APL  features.  You  need  only  assure  logical 
equivalence.  The  software  design  shows  where  the  detailed  programming 
for  these  items  belongs  in  the  general  scheme  of  things.  The  notion 

of  logical  equivalence  of  the  implemented  program  applies  as  well  to 
software  design  features  as  it  does  to  APL  description  features, 

4)  Memory  access  (MAC)  is  not  shown  explicitly  as  a module  of  the 
design.  Some  form  of  simulated  memory  access  is  certainly  implied  by 
provisions  for  instruction  fetching;,  retrieving  an  operand  for  adding 
or  subtracting  and  for  loading  or  storing  the  accumulator.  Also 
instruction  tracing  is  not  shown  as  an  explicit  module.  It  may  be 
highly  desirable  to  implement  these  features,  and  others,  as  explicit 
subprograms.  They  are  not  shown  explicitly  in  the  sample  design 
given  because  it  was  not  deemed  necessary  to  show  any  logical  design 
for  them,  (Perhaps  such  designs  would  be  included  in  the  next  step 
for  "top  down"  design  advocates.) 


SMALL_CCMPU"'ER_SIMUUTOR  i PROCEDURE; 

INITIALIZE  SIMULATOR; 

DO  WHILE  RUN  CODE  ■=- CONTINUE  EXECUTION  CODE; 

FETCH  INSTRUCTION; 

INCREMENT  LOCATION  COUNTER  PART  OF  PROGRAM  STATUS  WORD; 

IF  INSTRUCTION  IS  "HALT* INSTRUCTION  THEN  RUN  CODE  4?  HALT  TERMINATION  CODE; 
^ ELSE 

DO; 

DETERMINE  INSTRUCTION  CLASS,  INSTRUCTION  SUBCLASS  AND 
OPERAND  ADDRESS  FROM  INSTRUCTION; 

DO  CASE(  INSTRUCTION  CUSS  ) ; 

CASEOi 

CALL  PROCESS_LOAD_STCRE; 

CAS El i 

CALL  PROCESS_ADD_SUETRACT; 

CASE2 i 

CALL  PROCESS_BRANCH; 

CASE3; 

CALL  PROCESS_READ  WRITE  TRACE; 

END  CASE; 

END; 

IF  TRACE  IS  REQUESTED  THEN  PERFORM  TRACE; 

INCREMENT  INSTRUCTION  EXECUTION  COUNT; 

IF  INSTRUCTION  EXECUTION  COUNT  EXCEEDS  EXECUTION  LIMIT  THEN 
RUN  CODE  4 EXECUTION  LIMIT  TERMINATION  CODE; 

END; 

PRINT  SIMULATOR  STATUS  INFORMATION  UPON  TERMINATION; 

END  SMA  LL_CCM  PUTZR_S IMULATGR j 


PROCESS  LOAD  STORE*  PROCEDURE; 

DO  CASEX  INSTRUCTION  SUBCUSS  ); 

CASEOi 

RUN  CODE  ^SIMUUTOR  FAILURE  CODE; 

CAS El i 

STORE  ACCUMUUTOR  IN  MAIN  MEMORY( OPERAND  ADDRESS) ; 
CASE2 i 

LOAD  ACCUMUUTOR  FROM  MAIN  MEXORY( OPERAND  ADDRESS) ; 
CASE3I 

RUN  CODE  ^ INVALID  INSTRUCTION  CODE; 

END  CASE; 

RETURN  TC  CALLER; 

END  PRCCESS_LOAD_STOR  E ; 


I 


t 


s 
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PROCESS  ADD_SUETRACT  i PROC  EDUR  E ; 

RETRIEVE  OPERAND  FROM  MAIN  MEJ'XRY(  OPERAND  ADDRESS); 

DO  CASS(  INSTRUCTION  SUBCLASS  ); 

CASEOi 

RUN  CODE  ^ INVALID  INSTRUCTION  CODE; 

CASEli 

SUBTRACT  OPERAND  FROM  ACCUMULATOR  WITH.  RESULT  LEFT 
IN  ACCUMULATOR  AMD  CARRY  BIT  ASSIGNED  TO 
PROGRAM  STATUS  WORD; 

CASE  i 

ADD  OPERAND  TO  ACCUMULATOR  WITH  RESULT  LEFT  IN 
ACCUMULATOR  AND  CARRY  BIT  ASSIGNED  TO  PROGRAM 
STATUS  WORD; 

CASE3: 

RUN  CODE  ^-INVALID  INSTRUCTION  CODE; 

END  CASE; 

RETURN  TO  CAILER ; 

RND  PHOCESS_ADD_S  U3TRA CT ; 

PROCESS  BRANCH!  PROCEDURE; 

INTERPRET  OPERAND  ADDRESS  AS  BRANCH  TARGET; 

INTERPRET  INSTRUCTION  SUBCLASS  AS  INSTRUCTION  CONDITION  CODE; 

IF  INSTRUCTION  CONDITION  CODS  MATCHES  PROGRAM  STATUS  WORD 
CONDITION  CODE  THEN  ASSIGN  BRANCH  TARGET  TO  LOCATION 
COUNTER  PART  OF  PROGRAM  STATUS  WCRD; 

RETURN  TO  CALLER ; 

END  ?ROCESS_ERANCH; 

PROCESS  READ_WRITE_TRACE ; 

DETERMINE  MAIN  MEMORY  BLOCK  ADDRESS  AND  EXTERNAL  MEMORY  BLOCK 
ADDRESS  FROM  INSTRUCTION; 

DO  CASE(  INSTRUCTION  SUBCUSS  ) ; 

CASEOj 

. TURN  TRACE  REQUEST  ON; 

CAS  El  t 

TRANSMIT  64  CONTIGUOUS  WORDS  FROM  ADDRESSED  MAIN  MEMORY 
BLOCK  TO  ADDRESSED  EXTERNAL  MEMORY  BLOCK; 

CASE2 1 

TRANSMIT  ADDRESSED  EXTERNAL  MEMORY  BLOCK  TO  64  CONTIGUOUS 
WORDS  OF  ADDRESSED  MAIN  MEMORY  BLOCK; 

CASE3« 

TURN  OFF  TRACE  REQUEST; 

END  CASE 

RETURN  TO  CALLER; 

END  PROCESS  READ  WRITE_TRACE; 
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PROCESS  LOAD  STORE:  I ENTER 


iMSTRUCTIO 

SUBCUSS? 


RUN  CODE 

SIMUUTOR 

FAILURE 

CODE 


STORE 

ACCUMULATOR 
IN  MAIN 

MEMORY (OPERAND 
ADDRESS) 


^^RSTURN 
PROCESS  ADD  SUBTRACT:  / ENTER 


LOAD 



RUN  CODE 

ACCUMULATOR 

FROM  MAIN 

INVALID 

MEMORY ( OPERAND 

INSTRUCTION 

ADDRESS) 

CODE 

I! 


RETRIEVE  OPERAND 
FROM  MAIN  MEMORY 
(OPERAND  ADDRESS) 


RUN  CODE 

INVALID 

INSTRUCTION 

CODE 


SUBTRACT  OPERAND 
FROM  ACCUMULATOR 
WITH  RESULT  LEFT 
IN  ACCUMULATOR  AND 
CARRY  BIT  ASSIGNED 
TO  PROGRAM  STATUS 
WORD 


ADD  OPERAND 
TO  A CUNUUTOR 
WITH  RESULT  LIFT 
IN  ACCUMULATOR  AND: 
CARRY  BIT  ASSIGNED. 
TO  PROGRAM  STATUS  I 
WORD 


RUN  CODE 

INVALID 

INSTRUCTION 

CCDS 


RETURN 


t#v 


PROCESS  BRANCH:  ( ENTER 


INTERPRET  OPERAND  ADDRESS 
AS  BRANCH  TARGET 


INTERPRET  INSTRUCTION  SUBCLASS 
’AS  INSTRUCTION  CONDITION  CODE 


ASSIGN  BRANCH  TARGET 
TO  LOCATION  COUNTER  , 
PART  OF  PROGRAM  STATUS 
WORD 


PROCESS  REJSD  WRITE  TRACE: 


DETERMINE  MAIN  MEMORY  BLOCK  ADDRESS 
AND  EXTERNAL  MEMORY  BLOCK  ADDRESS 
FROM  INSTRUCTION 


'INSTRUCTION  \ 
CONDITION  CODE 
MATCHES  PROGRAM 
STATUS  WORD 
CONDITION  CODE. 


INSTRUCTION 

SUBCUSS? 


TURN  TRACE  TRANSMIT  64  CONTIGUOUS 

REQUEST  ON  WORDS  FROM  ADDRESSED 

MAIN  MEMORY  TO  ADDRESSED 
EXTERNAL  MEMORY  BLOCK 


RANSMIT  ADDRESSED 
(EXTERNAL  MEMORY  BLOCK 
(TO  64  CONTIGUOUS  WORDS 
OF  ADDRESSED  MAIN 
MEMORY  BLOCK 


TURN  OFF 
TRACE  REQUS-S! 


! ) 


\ 
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APPENDIX  F - BASIC  DESCRIPTION  OF  DESIGN  PROBLEM 

Assembler  Language  Translator  Design  Problem 


An  assembler  language  for  the  hypothetical  "small  computer" 

(Small  Computer  Assembler  Language  - SCAL)  is  outlined  below.  You 
will  be  given  a software  design  assignment  requirement  for  a two 
pass  translator  for  SCAL,  One  pass  of  the  translator  is  to  be 
designed  with  one  of  the  two  software  design  methods  presented  in 
class  (flowcharts  and  PDL's).  The  other  pass  is  to  be  designed 
using  the  other  method. 

Half  the  class  will  be  required  to  use  flowcharts  for  the 
first  pass  and  half  the  class  will  be  required  to  use  PDL.  The 
designs  from  those  students  consenting  to  participate  in  the 
software  design  study  being  conducted  this  semester  will  be  used 
(anonymously)  for  part  of  this  study. 

Written  requirements  for  each  pass  are  given  on  a separate 
handout.  The  pass  one  assignment  will  be  distributed  in  class 
Feb.  10  and  will  be  due  Feb.  15 . (Turn  it  in  to  Dr.  Fisher’s 
secretary.)  The  pass  two  assignment  is  to  be  picked  up  in  Dr.  Fisher's 
office  when  you  turn  in  the  pass  one  part.  The  pass  two  design  will 
be  due  Feb.  17. 


SCAL  features: 

1)  Symbolic  machine  instructions  with  instruction  formats 

Optional  label  HALT  Optional  symbolic  or  absolute  address 

Symbolic  or  absolute  address 


LOAD 

STORE 


ADD 

SUB 

B 


Numeric  condition  code(0-3) , symbolic  or 

absolute  address 


\ »■ 


READ 


2)  Constant  definitions  with  formats 


Symbolic  or  absolute  address 
* 


Optional  label  DC 

M M DIO 


Integer  constant 


Symbolic  or  absolute 
memory  block  address 


symbolic  or  absolute 
external  store  address 


Note:  Did  is  intended  for  defining  the  addresses  for  READ  or  WRITE 
oprrations.  Ordinarily  a DIO  would  be  referenced  only  by 
a READ  or  WRITE.  Ordinarily  r/fey mbolic  reference  in  a READ  cr 
WRITE  would  be  a reference  to  a DIO,  Neither  of  these  are 
required,  however. 
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3)  Assembler  instructions  with  formats 


Required  label 


PROG  Absolute  address  (starting  location  of  program) 

EQUATE  Absolute  address  (address  value  of  label  symbol) 

LOC  Absolute  address  (reassigns  assembler  location 

counter  for  location  of 
following  instructions) 


Optional  label  DS 


Positive  integer 


value 

(number  of  words  of 
storage  reserved) 


END  Symbolic  or  absolute  address 

(address  of  first  instruction 
to  be  executed) 


The  symbolic  addresses  are  very  simple  in  form  and  do  not  allow 
address  expressions  such  as  *+3  or  SUM+2.  Neither  are  literal  operands 
allowed.  All  absolute  addresses  or  numeric  constants  are  assumed  to 
be  in  decimal  form. 


Comments  and  observations  about  the  assignment: 

You  are  encouraged  to  design  your  logic  at  a level  similar  to  that 
displayed  in  the  software  design  illustrations.  For  example,  don't  be 
concerned  with  the  detailed  logic  for  " recognizing"  a valid  symbol  or 
constant  (lexical  analysis  or  scanning) . You  should  be  able  to  complete 
your  design  in  8j  x 11  pages  or  less. 

Assume  that  symbol  table  design  and  processing  algorithms  already 
exist;  that  is  to  say,  do  not  attempt  to  design  them.  Further  assume 
that  the  symbol  table  structure  is  very  simple  in  that  no  "type"  codes 
are  required  - just  symbols  and  absolute  addresses. 

Neither  the  PDL  nor  the  flowchart  design  method  provides  for  data 
structure  definition.  For  the  most  part  data  structure  requirements 
for  the  assembler  are  elementary  and  may  be  implied  by  appropriate 
wordage  and  symbolism.  If  certain  tables  or  data  structures  used  in 
your  design  are  sufficiently  complex  that  separate  descriptions  of  them 
are  needed,  then  provide  them. 
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i Hex  r*v*t  IKK 
koc,  iMftH  liCNTL 


44  £ ooo 

45  eooo 
44  2000 
4l  l Ooo 
49  z ooo 
U\  I ooo 
4A  Zooo 

46  I ooo 
4C  2ooo 
4D  I ooo 
4E  20oo 
4F  5 ooo 
70  4ooo 

11  l ooo 

12  2 ooo 
15  4 ooo 
1H  I ooo 
15  2 ooo 

14  4 ooo 
11  I ooo 
72  2ooo 
H 4 ooo 
7A  l ooo 

15  5 ooo 
TC  2ooo 


XNPUT  XMA«e 

U»C  05»  AteK 

PRj6&  I0O 
1337  READ  XAOl 
Read  XAD2 
L*AD  T3TI 
St<*RE  TEST 


.STORE  LB  l 
LOAD  LBtl 
3v^RE  LB4 
LMO  ZSfctf 
3WM  C 
LB  2 L#Ap  A 
LB4  50B  6 

ADD  C 
StyRS  C 
LOAD  452 
ADD  C6ME 
STORE  LB2 
Load  l&4 
ADD  0NS 
5T<$Re  LB* 
L0AD  test 
Add  (tn£ 
store  test 
BR  O L33 
LpAt>  MAX 


I;xm?LS  ?At-Si 


mx  xMA*,g  bm  tsip-jt  x>sa<ss 
[loc.x»u?X|  J Cntc  I LOC  , OP  . r< 


ID  4000 

ie  iMo 


20  oa&e 
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SfODSUOM  PAGE  RUMfaMOT  FILMED 

APPENDIX  6 - DESCRIPTION  OF  PASS  1 DESIGN  PROBLEM 
Assembler  language  Translator  Design  Problem,  Pass  1 

Name i Design  Method i Flowchart  / PDL 

Pass  one  has  the  following  responsibilities! 

To  read,  process  and  copy  the  SCAL  program  to  a secondary  storage 
file  in  the  form  suggested  in  the  example  handout. 

To  fill  in  the  symbol  table  for  defined  symbols. 

To  detect  invalid  operation  codes.  (Substitute  a "HALT  0* image 
for  these,) 

To  detect  multiple  definitions  of  a symbolic  address. 

To  detect  "out  of  range  address”  situatuens. 

To  detect  other  errors  and  take  error  action  consistent  with  your 
overall  software  design. 


Note  li  The  copy  of  the  SCAL  program  will  servd  as  input  to  the  second 
pass , 

Note  2 « Assume  that  all  necessary  information  is  passed  in  a global 
sense  to  pass  twos  that  is,  it  isn't  necessary  to  identify 
explicit  parameters. 

Note  3*  Make  sure  you  fill  out  and  turn  in  the  time  questionaire  if 

you  are  participating  in  the  software  design  experiment.  This 
information  is  needed  for  the  study. 
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APPENDIX  H - DESCRIPTION  OF  PASS  2 DESIGN  PROBLEM 
Assembler  Language  Translator  Design  Problem,  Pass  2 

Namei  Design  Method » Flowchart  / PDL 


!^.ss  two  has  the  follwoing  responsibilities j 

To  generate  absolute  object  code  in  the  format  indicated  below  on 
a secondary  file. 

To  create  a prihted  form  of  the  orginal  program  with  machine  code 
information  and  error  codes  as  suggested  by  the  example  handout. 

To  detect  undefined  symbolic  addresses.  (Substitute  a zero  for  such 
an  address  in  the  generated  code.) 

To  detect  "out  of  range"  absolute  addresses  and  constants. 

To  detect  errors  that  shouldn't  occur  if  the  first  pass  is  working 
correctly. 

To  detect  other  errors  and  take  error  action  consistent  with  your 
overall  software  design. 


Format  of  generated  code  - 40  word  records 


1st  word 

2nd  word 
Words  3 to  40  - 


absolute  addres  where  machine  language  text 
is  to  be  loaded 

number  of  wortfe  of  machine  language  text  following 
machine  language  text 


"trailer"  record 
1st  word  - 
2nd  word  7 
3rd  word  - 


"0" 

"0" 

Absolute  address  of  starting  instruction 


Notei  Make  sure  you  fill  out  and  turn  in  the  time  questionaire 
This  information  is  needed  for  the  software  design  study. 
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APPENDIX  I - TIME  EXPENDITURE  QUESTIONNAIRE,  DESIGN  PHASE 

TIME  QUESTION A IRE 
ASSEMBLER  DESIGN  PROBLEM,  PASS  1 


(Leave  blank) 

Please  record  the  time  you  have  spent  on  your  class  project  in  the  last  week, 
excluding  time  spent  in  class.  Record  time  in  hours  and/or  minutes  by  category 


hrs  : min 


Time  spent  in: 

Preparing  initial  design  

Revising  design  

Hand-checking  design  

Preparing  documentation  

Just  thinking  about  project  . . . . 
Other  (indicate  nature  of  activity) 


This  information  is  solely  for  experimental  purposes;  your  responses  will  in 
no  way  affect  your  course  grade.  Please  be  as  accurate  as  possible. 
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APPENDIX  J - PDL-FLOWCHART  QUESTIONNAIRE 


n 


M 


Name: 


INSTRUCTIONS 


This  questionnaire  Is  designed  solely  for  experimental  purposes; 
your  responses  will  in  no  way  affect  your  course  grade.  The  goal 
of  this  questionnaire  is  to  provide  us  with  an  objective,  unbiased 
comparison  of  flowcharts  and  program  design  languages  (PDLs)  as 
methods  for  the  development  and  documentation  of  software  designs. 
Please  complete  these  questions  honestly  and  thoughtfully,  so 
that  your  answers  reflect  your  opinions  about  these  methods 
based  on  the  experience  you  have  had  with  them. 

Most  of  the  questions  which  follow  ask  you  to  compare  the  two 
methods  by  checking  a box  on  a scale.  In  every  case,  check  one 
and  only  one  box,  and  place  your  checks  within  the  box,  rather 
than  on  boundaries  between  boxes. 


Code: 


1.  From  the  viewpoint  of  the  software  designer  who  must  use  flowcharts 
or  a program  design  language  to  document  his  design,  how  do  the  two 
ccmpa re? 


Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 

slightly 

hotter 

POL 

slightly 

better 

POL 

moderately 

better 

POL 

much 

better 

POL 

absolutely 

better 

2.  From  the  viewpoint  of  the  programmer  who  must  understand  and  implement 
a design  expressed  in  either  a program  design  language  or  flowcharts, 
how  do  the  two  compare? 


POL 

absolutely 

better 

POL 

much 

better 

POL 

modera tely 
better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

■ 

3.  How  do  the  methods  compare  with  respect  to  helping  the  software  designer 
understand  his  problem? 


Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 
si  Ightly 
better 

POL 

slightly 

better 

POL 

moderately 

better 

POL 

much 

better 

POL 

absolutely 

better 

4.  With  respect  to  encouraging  the  use  of  "good"  design  practices? 


PUL 

absolutely 

better 

POL 

much 

better 

POL 

moderately 

better 

POL 

slightly 

better 

Flowchart 
* slightly 
better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

5.  Ease  of  preparation  of  design  documentation? 


Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 
si Ightly 
better 

POL 

slightly 

better 

POL 

moderately 

better 

POL 

much 

better 

POL 

absolutely 

better 

6.  Ease  of  modification  of  design  documentation? 


PUL  POL 

absolutely  much 

better  better 


PUL  POL  Flowchart  Flowchart  Flowchart  Flowchart 

moderately  slightly  slightly  moderately  much  absolutely 
better  better  better  better  better  better 


7.  Ease  of  modification  of  design? 


Flowchart  Flowchart  Flowchart  Flowchart  POL  POL  POL  POL 

absolutely  much  moderately  slightly  slightly  moderately  much  absolutely 

better  better  better  better  better  better  better  better 


Ease  of  expression  of  control  flow  by  designer? 


POL  POL  POL  POL  Flowchart  Flowchart  Flowchart  Flowchart  1 

absolutely  much  moderately  slightly  slightly  moderately  much  absolutely  ■ 

better  better  better  better  better  better  better  better 


Ease  of  expression  of  data  flow? 


Flowchart  Flowchart  Flowchart  Flowchart  POL  POL 

absolutely  much  moderately  slightly  slightly  moderately 

better  better  better  better  better  better 


POL 

absolutely 

better 


10.  From  the  viewpoint  of  the  programmer  who  must  understand  and  implement 
the  design,  how  do  the  methods  compare  on  comprehensibility? 


POL 

much 

better 

POL 

moderately 

better 

POL 

slightly 

better 

Flowchart 

Si ightly 
better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

How  do  they  compare  on  ease  of  reading  (i.e.,  effort  involved,  speed,  etc.)? 

Flowchart  Flowchart  Flowchart  Flowchart  POL  POL  POL  POL 

absolutely  much  moderately  slightly  slightly  moderately  much  absolutely 

better  better  better  better  better  better  better  better 


12.  Ease  of  understanding  control  flow? 


PUL 

absolutely 

tetter 

POL 

much 

better 

POL 

modera tely 
better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

13.  Ease  of  understanding  data  flow? 


Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 

slightly 

better 

POL 

slightly 

better 

POL 

moderately 

better 

POL 

much 

better 

POL 

absolutely 

better 

14.  Ease  of  implementing  in  a programming  language? 


POL 

absolutely 

better 

POL 

much 

better 

POL 

moderately 

better 

POL 

si ightly 
better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

15.  As  the  designer  develops  his  design,  he  may  notice  that  it  contains 
errors  or  undesirable  properties.  How  do  the  documentation  methods 
comDare  with  respect  to  helping  the  designer  detect  such  problems  in 
his  design? 


Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 

slightly 

better 

POL 

slightly 

better 

POL 

moderately 

better 

POL 

much 

better 

POL 

absolutely 

better 

16.  How  do  the  methods  compare  with  respect  to  helping  the  designer  notice 
that  the  same  operation  is  being  performed  redundantly  at  several  points 
in  the  system  he  is  designing? 


POL 

absolutely 

better 

POL 

much 

better 

POL 

moderately 

better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

J-4 


, 


T- 


i 


<* 


n * 

Li'*-' 


17. 


With  respect  to  helping  him/her  notice  that  a module  is  too  big,  and 
should  be  broken  into  several  modules? 


Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 

slightly 

better 

POL 

slightly 

better 

PDl 

moderately 

better 

POL 

much 

better 

POL 

absolutely 

better 

18.  Helping  him/her  notice  that  a module  is  logically  too  complex? 


19. 


POL 

absolutely 

better 

POL 

much 

better 

POL 

modera tely 
better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

a modu' 

e has  too  many 

or  too  few  submodules? 

Flowchart 

absolutely 

better 

Flowchart 

much 

better 

Flowchart 

moderately 

better 

Flowchart 

slightly 

better 

POL 

slightly 

better 

PDL 

modera tely 
better 

POL 

much 

better 

POL 

absolutely 

better 

20. 


That  a module  contains  several  logically  unrelated  functions  (i.e.,  that 
the  module  is  not  cohesive)? 


PUL 

absolutely 

better 

POL 

much 

better 

POL 

moderately 

better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

21.  That  a module  has  not  been  defined  in  enough  detail? 


i 

Flowchart 

Flowchart 

Flowchart 

Flowchart 

POL 

POL 

POL 

POL 

absolutely 

much 

moderately 

slightly 

slightly 

moderately 

much 

absolutely 

* 

better 

better 

better 

better 

better 

better 

better 

better 

J 

22.  That  a module  has  been  defined  in  too  much  detail? 


PUL 

absolutely 

better 

POL 

much 

better 

PDL 

moderately 

better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 
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28.  Questions  16  through  27  have  asked  you  how  flowcharts  and  program  design 
languages  compare  with  respect  to  helping  the  designer  notice  a number 
of  specific  problems  in  his/her  design.  In  general,  did  you  find  these 
questions  (pick  one): 

( ) easy  to  answer. 

( ) difficult  to  answer  because  you  don't  see  any  difference 
between  the  documentation  methods  with  respect  to  these 
criteria. 

( ) Difficult  to  answer  because  you  couldn't  visualize  the 
kinds  of  design  errors  and  problems  referred  to  In  the 
questions. 


29.  Overall,  how  do  you  think  the  two  documentation  methods  compare? 


POL 

absolutely 

better 

POt 

■uch 

better 

POL 

moderately 

better 

POL 

slightly 

better 

Flowchart 

slightly 

better 

Flowchart 

moderately 

better 

Flowchart 

much 

better 

Flowchart 

absolutely 

better 

30.  Are  there  any  other  documentation  methods  you  think  are  better  than 
either  of  these? 
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31. 


Please  write  any  other  comments  you  have  on  these  two  methods  (or 
others  you  think  are  superior).  In  using  both  methods  for  a design 
problem,  you  may  very  well  have  acquired  some  insights  we  don't  have. 
Therefore,  please  comment  on  even  obvious  things  if  they  have  not  been 
addressed  in  this  questionnaire. 


APPENDIX  K - SIMULATOR  DESIGN,  FLOWCHART  FORM 


Name 


COMSC  5113  SIMULATOR  DESIGN 


Sp-ing  1977 


Explanatory  notes: 

1)  The  location  counter  in  this  design  generally  points  to  the  next 
Instruction  in  contrast  to  the  current  instruction  as  stated 

in  the  microNova  manual.  This  tends  to  simplify  the  design  but 
has  implications  concerning  tracing  and  dumping. 

2)  There  are  no  deliberate  errors  in  the  design.  If  errors  are 
detected  or  clarification  of  microNova  functional  characteristics 
occurs  then  revision  information  will  be  distributed  in  the  form 
of  memoranda. 

3)  Input/output  devices  are  not  fully  simulated.  A simulator  block 
input  and  block  output  scheme  is  used  instead. 

4)  Console  operations  are  not  simulated  in  the  design. 

5)  Half-  of  the  class  is  receiving  a PDL  design  and  half  of  the  class 
is  receiving  a flowchart  design.  These  two  designs  contain  equivalent 
information.  If  you  are  participating  in  the  software  design  study 
you  are  requested  to  use  only  the  design  given  you  for  your 
simulator  implementation. 

6)  Questions  concerning  the  design  should  be  directed  to  Dr.  J.R. 

Van  Doren.  Dr.  D.D.  Fisher  should  be  consulted  concerning  questions 
about  microNova  functional  characteristics. 
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APPENDIX  L - SIMULATOR  DESIGN,  PDL  FORM 


Na me 


COMSC  5113 


SIMULATOR  DESIGN 


Spring  1977 


Explanatory  notes: 


1)  The  location  counter  in  this  design  generally  points  to  the  next 
instruction  in  contrast  to  the  current  instruction  as  stated 
in  the  microNova  manual.  This  tends  to  simplify  the  design  but 
has  implications  concerning  tracing  and  dumping. 


2)  There  are  no  deliberate  errors  in  the  design.  If  errors  are 
detected  or  clarification  of  microNova  functional  characteristics 
occurs  then  revision  information  will  be  distributed  in  the  form 
of  memoranda. 


3)  Input/output  devices  are  not  fully  simulated, 
input  and  block  output  scheme  is  used  instead. 


A simulator  block 


4)  Console  operations  are  not  simulated  in  the  design. 


5)  Half  of  the  class  is  receiving  a PDL  design  and  half  of  the  class 
is  receiving  a flowchart  design.  These  two  designs  cbntain  equivalent 
information.  If  you  are  participating  in  the  software  design  study 
you  are  requested  to  use  only  the  design  given  you  for  your 
simulator  implementation. 


6)  Questions  concerning  the  design  should  be  directed  to  Dr.  J.R. 
Van  Doren.  Dr.  D.D.  Fisher  should  be  consulted  concerning  questions 
about  microNova  functional  characteristics. 
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MICRON  OVA  SIMULATOR; 

PROCEDURE | 

CALL  It!lTIALIZE_SI!iL'LATORj 
DO  WHILE  TERMINATION  CODE  = 0; 

FETCH  INSTRUCTION; 

INCREMENT  LOCATION  COUNTER; 

IF  PRIOR  INSTRUCTION  WAS  "KASK  OUT"  THEN 
TURN  ON  PRIORITY  KASK  UPDATE  SWITCH; 

CALL  EXECUTE  INSTRUCTION; 

IF  PRIORITY  KASK  UPDATE  SWITCH  IS  ON  THEN 
DO; 

CURRENT  PRIORITY  KASK  = NEW  PRIORITY  KASK; 

TURN  OFF  PRIORITY  KASK  UPDATE  SWITCH; 

END; 

INCREMENT  CURRENT  CYCLE  COUNT  BY  INSTRUCTION  CYCLE  COUNT; 

IF  CURRENT  CYCLE  COUNT  > CYCLE  LIMIT  THEN 
TERMINATION  CODS  = EXCESSIVE  CYCLE  CODE; 

IF  TRACE  IS  REQUESTED  THEN 
DO; 

PERFORM  INSTRUCTION  TRACE; 

DECREMENT  TRACE  COUNT  BY  1; 

/IF  TRACE  COUNT  = 0 THEN  TURN  OFF  TRACE  REQUEST; 

END; 

IF  SKIP  INDICATOR  IS  ON  THEN 
DO; 

TURN  OFF  SKIP  INDICATOR; 

INCREMENT  LOCATION  COUNTER; 

END; 

IF  REAL  TIME  CLOCK  IS  ENABLED  THEN 
DO; 

DECREMENT  CLOCK  CYCLE  COUNT  BY  INSTRUCTION  CYCLE  COUNT; 

IF  CLOCK  CYCLE  COUNT  £0  THEN 
DO; 

SETUP  INTERRUPT  CONDITION; 

RESET  REAL  TIME  CLOCK; 

END; 

END; 

INSTRUCTION  CYCLE  COUNT  = 0; 

DO  WHILE  INTERRUPT  CYCLE  COUNT  < CURRENT  CYCLE  COUNT; 

IF  DEVICE  CODE  IS  ON  AND  CURRENT  PRIORITY  MASK  ALLOWS 

DEVICE  INTERRUPT  THEN  SETUP  INTERRUPT  FRCM  INTERRUPT  LIST; 
UPDATE  INTERRUPT  CYCLE  COUNT  TO  NEXT  INTERRUPT; 

END; 

IF  AN  INTERRUPT  IS  REQ3STED  AND  INTERRUPTS  ARE  ENABLED 
AND  INSTRUCTION  EXECUTED  WAS  NOT  "INTERRUPT  ENABLE" 

THEN  CALL  PROCESS  INTERRUPT; 

END; 

PRINT  SIMULATOR  STATUS  INFORMATION  UPON  TERMINATION; 

END  MICRONOVA  SIMULATOR; 
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INITIALIZE  SIMULATOR;  PROCEDURE; 

LOAD  MICRON OVA  PROGRAM  FROM  CARD  DECK ; 

INITIALIZE' LOCATION  COUNTER  AMD  MAIN  MEMORY  SIZE; 

INITIALIZE  ALL  INTERRUPT  FLAGS,  INPUT/OUTPUT  FUGS,  PRIORITY  MASKS, 
SIMUUTOR  CONTROL  FUGS,  ETC.; 

INITIALIZE  TERMINATION  CODE,  CURRENT  CYCLE  COUNT,  INSTRUCTION  CYCLE 
COUNT  TO  ZERO} 

INITIALIZE  CYCLE  COUNT  TABLE  AND  CYCLE  LIMIT; 

INITIALIZE  INTERRUPT  LIST  FOR  INTZRRU/T  SIMULATION; 

INITIALIZE  REAL  TIME  CLOCK  RESET  VALUE  AND  RANDOM  RESET  VALUE; 
PERFORM  MISCELLANEOUS  INITIALIZATION; 

RETURN  TO  CALLER; 

END  INITIALIZE  SIMULATOR; 


EXECUTE  INSTRUCTION;  PROCEDURE; 

CYCLE  TABLE  INDEX  = 1 ; 

IF  LEFTMOST  BIT  OF  INSTRUCTION  IS  '1*  THEN  CALL  TWO_ACCUKUUTOR__INST; 
ELSE'lF  NEXT  TWO  BITS  C?  INSTRUCTION  ARE  NOT  ' 11'  THEN 
CALL  MEMORY_REFERENCE_INST; 

ELSE 

DO; 

ASSIGN  RIGHTMOST  SIX  BITS  OF  INSTRUCTION  TO  DEVICE  CODE; 
IF  DEVICE  CODE  = 1 THEM  CALL  STACK_OR _MUL_DIV; 

ELSE  IF  DEVICE  CODE  = 62  THEN  CA LL~TRA CE_DUKP ; 

ELSE  IF  DEVICE  CODE  = 63  THEN  CALL  CPU  FUNCTION; 
ELSE  CALL  10  INST; 

END;  “ 

INCREMENT  INSTRUCTION  CYCLE  COUNT  BY  CYCLE  TABLE( CYCLE  TABLE  INDEX); 
RETURN  TO  CALLER; 

END  EXECUTE  INSTRUCTION; 


PROCESS  INTERRUPT;  PROCEDURE; 

DISABLE  INTERRUPTS; 

INCREMENT  INSTRUCTION  CYCLE  COUNT  BY  INTERRUPT  CYCLE  CCUNT; 
STORE  LOCATION  COUNTER  IN  ZERO  POSITION  OF  MAIN  MEMORY  VECTOR; 
DO  REQUEST  LINE  = 3 TO  1 BY  -1; 

IF  INTERRUPT  REQUEST(REQUEST  LINE)  IS  ON  THEN 
DO; 

EFFECTIVE  ADDRESS  = REQUEST  LINE; 

CALL  FIND  INDIRSCT_ADDRSSS; 

ASSIGN  EFFECTIVE  ADDRESS  TO  LOCATION  COUNTER; 

RETURN  TO  CALLER; 

END; 

END; 

TERMINATION  CODE  = INTERRUPT  SIMUUTION  FAILURE  CODE; 

RETURN  TO  CALLER; 

END  PROCESS  INTERRUPT; 


HB-iORY  REFERENCE  INST  i PROCEDURE; 

C/.LL  COMPUTE  EFFECTIVE  ADDRESS  j 

IF  BITS  1 AND  2 OF  INSTRUCTION  ARE  ’OO*  THEN  CALL  NO  ACCUMULATOR  INST; 
ELSE  CALL  ONE  ACCUMUL\TOR  INST; 

RErURN  TO  CALLER ; 

END  MEMORY  REFERENCE  INST; 


ONE  ACCUKULATOR_TNST;  PROCEDURE; 

“ EXTRACT  0?  CODE  AND  ACC'/;: 

DO  CASE  (OP  CODE) ; 

CASEOj 

TERMINATION  CODE  = SIHILATCR.'  FAILURE  CODE; 

CAS El i 
DO; 

CYCLE  TABLE  INDEX  = 2 ; 

LOAD  ACCUMULATOR (ACC#)  FROM  MAIN  MEMORY (EFFECTIVE  ADDRESS) ; 

END; 

CASE2i 

DO; 

CYCLE  TABLE  INDEX  =3; 

STORE  ACCUMULATOR (ACC/O  IN  MAIN  MEMCRY( EFFECTIVE  ADDRESS); 
END; 

CASE3i  • 

TERMINATION  CODE  = SIMULATOR  FAILURE  CODE; 

END  CASE; 

RETURN  TO  CALLER; 

END  ON E_A CCUMU LAT OR  INST; 


N 0_A C CUMU LA TOR_I N ST j PROCEDURE; 

EXTRACT  OP  CODE; 

DO  CASE  ( OP  CODE  ) ; 

CASEOj 

DO; 

CYCLE  TABLE  INDEX  = 2 ; 

ASSIGN  EFFECTIVE  ADDRESS  RO  LOCATION  COUNTER; 
END; 

CASElt 

DO; 

CYCLE  TABLE  INDEX  = 3; 

ASSIGN  LOCATION  COUNTER  TO  ACCUMULATOR^) ; 
ASSICN  EFFECTIVE  ADDRESS  TO  LOCATION  COUNTER; 
END; 

CASE2 i 
DO; 

CYCLE  TABLE  INDEX  = 4; 

INCREMENT  MAIN  UEKCRY( EFFECTIVE  ADDRESS)  BY  1; 
IF  RESULT  IS  ZERO  THEN  TURN  ON  SKIP  INDICATOR; 

END; 

CASE3I 

DO; 

CYCLE  TABLE  INDEX  = 4; 

DECREMENT  MAIN  I .ETiCRY(  EFFECT "VE  ADDRESS)  BY  1; 
IF  RESULT  IS  ZERO  THEN  TURN  ON  SKIP  INDICATOR; 
END; 

END  CASE; 

RETURN  TO  CALLER;  L-4 

SND  NO  ACCUMULATOR  INST; 
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COMPUTE  EFFECTIVE  ADDRESS « PROCEDURE; 

DECOMPOSE'  11“aDDR‘SS  BITS  INTO  INDIRECT  BIT,  INDEXING  CODE, 

SIC NED  !)ISPIACS-:SNT,  UNSIGNED  DISPLACEMENT ; 

DO  CASE  (INDEXING  CODS); 

CASrX)  i 

EFFECTIVE  ADDRESS  = UNSIGNED  DISPLACEMENT; 

CAS  El  t 

EFFECTIVE  ADDRESS  = LOCATION  COUNTER  - 1 + SIGNED  DISPLACEMENT; 
CASES i 

EFFECTIVE  ADDRESS  = ACCUMULATOR^)  + SIGNED  DISPLACEMENT; 

CASE3« 

EFFECTIVE  ADDRESS  = ACCUMULATOR (3)  + SIGNED  DISPLACEMENT; 

END  CASE; 

RSrAIN  RIGHTMOST  15  BITS  OF  EFFECTIVE  ADDRESS; 

IF  INDIRECT  BIT  IS  ON  THEN  CALL  FIND_INDIRECT_ADDRESS ; 

RETURN  TO  CALLER; 

END  COMPUTE  EFFECTIVE  ADDRESS; 


FIND  INDIRECT_ADDRESS  i PROCEDURE; 

INITIALIZE  REFERENCE  COUNT  TO  2 ; 

DO  WHILE  REFERENCE  COUNT  £ 15; 

INCREMENT  INSTRUCTION  CYCLE  COUNT  BY  INDIRECT  ADDRESS  CYCLE  COUNT; 
RETRIEVE  INDIRECT  WORD  FROM  MAIN  MEMORY (EFFECTIVE  ADDRESS); 

IF  16  < EFFECTIVE  ADDRESS  < 31  THEN 
DO; 

IF  EFFECTIVE  ADDRESS  5:23  THEN  INCREMENT  INDIRECT  WORD  BY  1; 
ELSE  DECREMENT  INDIRECT  WORD  BY  1; 

STORE  INDIRECT  WORD  IN  MAIN  MEMCRY( EFFECTIVE  ADDRESS); 
INCREMENT  INSTRUCTION  CYCLE  COUNT  BY  AUTO  INDEX  CYCLE  COUNT; 
INCREMENT  REFERENCE  COUNT  BY  1 ; 

IF  REFERENCE  COUNT  > 15  THEN 
DO; 

TERMINATION  CODS  = INDIRECT  OVERFLOW  CODS; 

RETURN  TO  CALLER; 

END; 

END; 

ASSIGN  RIGHTMOST  15  BITS  OF  INDIRECT  WORD  TO  EFFECTIVE  ADDRESS ; 

IF  LEFTMOST  BIT  OF  INDIRECT  WORD  = 0 THEN  RETURN  TO  CALLER; 
INCREMENT  REFERENCE  COUNT  BY  2; 

END; 

TERMINATION  CODE  = INDIRECT  OVERFLOW  CODE; 

RETURN  TO  CALLER; 

END  FIND  INDIRECT  ADDRESS; 
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TWO  ACCUKULATOR_INSTR:  PROCEDURE; 

“ decckpcke  Instruction  into  source  acc//,  destination  acc#,  op  code, 

SHIFT  ACTION,  CARRX3IT  ACTION,  RESULT  ACTICN,  SKI?  ACTION » 

IF  RESULT  ACTICN  \_o)aND  SKIP  ACTION  = 0 THEN 
DO}  \ 

CALL  TRAP  INST; 

CYCLE  TAEL"  INDEX  = 4; 

RETURN  TO  CALLER; 

END; 

SET  CARRY  BIT  ACCORDING  TO  CARRY  BIT  ACTION; 

OPERAND  =A CCU;  1UIATCR (SOURCE  ACC,?)  ; 

DO  CASE  (OP  CCDS) ; 

CAS  BO  i 

PERFORM  COMPLEMENT  OF  DPERAND  WITH  RESULT  ASSIGNED  TO  SHIFTER; 
CASE1 i 

PERFORM  TWO'S  COMPLEMENT  NEGATION  OF  OPERAND  WITH  RESULT 
ASSIGNED  TO  SHIFTER; 

CASES: 

ASSIGN  OPERAND  TO  SHIFTER; 

CASE3 : 

ADD  1 TO  OPERAND  WITH  RESULT  ASSIGNED  TO  SHIFTER; 

CASEJ4: 

ADD  COMPLEMENT  OF  OPERAND  TO  ACCUHULATOR(DESTINATION  ACC#)  WITH 
RESULT  ASSIGNED  TO  SHIFTER; 

CASE5: 

SUBTRACT  OPERAND  FROM  ACCUMULATOR (DESTINATION  ACC#)  WITH 
RESULT  ASSIGNED  TO  SHIFTER; 

CASE6: 

2DD  OPERAND  TO  ACCUMULATOR (DESTINATION  A CO/)  WITH  RESULT 
ASSIGNED  TO  SHIFTER; 

CASE7t 

LOGICALLY  "AND"  OPERAND  BITS  WITH  ACCUMULATOR (DESTINATION  ACC#) 
BITS  WITH  RESULT  ASSIGNED  TO  SHIFTER; 

END  CASE; 

SET  LEFTMOST  BIT  OF  SHIFTER  ACCORDING  TO  FUNCTIONAL  SPECIFICATIONS ; 
PERFORM  SPECIFIED  SHIFT  ACTION  ON  SHIFTER ; 

SET  SKIP  INDICATOR  ACCORDING  TO  SKIP  ACTION  SPECIFICATIONS; 

IF  RESULT  ACTION  = 1 THEN  A 6CUMULA TOR ( D ESTI NA TI ON  ACC#)  = 

RIGHTMOST  16  BITS  OF  SHIFTER; 

EISE  IF  SKIP  ACTICN  = 0 THEN  TERMINATION  CODE  = SIMULATOR  FAILURE  CODE; 
CYCLE  TABLE  INDEX  = 1 | 

RETURN  TO  CALLER; 

END  TWO  ACCUMULATOR  INSTR ; 


TRAP_INST : PROCEDURE; 

ASSICN  LOCATION  COUNTER  TO  MAIN  MEMORY (38) ; 
EFFECTIVE  ADDRESS  = 39; 

CALL  FIND  INDIRECT.  ADDRESS; 

ASSIGN  EFFECTIVE  ADDRESS  TO  LOCATION  COUNTER ; 
RETURN  TO  CALLER; 

END  TRAP  INST; 
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STACK_CR  fiUL  DIV;  PROCEDURE; 

IF  MULTIPLY  ch  divide  instruction  then 
DO; 

IF  DIVIDE  INSTRUCTION  THEM 
DO; 

IF  ACCUMULATOR (0)  > ACCUMULATOR^)  THEM 
DO; 

CARRY  BIT  = 1; 

OVERFLOW  CODE  = 1; 

RETURN  TO  CALLER; 

END; 

PERFORM  DIVISION  OF  CATENATED  ACCUMULATORS  0 AND  1 
BY  ACCUMULATOR^) ; 

ASSIGN  QUOTIENT  AND  REMAINDER  TO  ACCUMULATORS  0 AND  1, 
RESPECTIVELY; 

RETURN  TO  CALLER; 

END; 

PERFORM  MULTIPLICATION  OF  ACCUMULATOR(l)  BY  ACCUMULATOR^)  WITH 
ACCUMULATOR ( 0 ) ADDED  TO  RESULT; 

ASSIGN  RESULT  TO  ACCUMULATORS  0 AND  1 ; 

RETURN  TO  CALLER; 

END; 

DECOMPOSE  INSTRUCTION  INTO  STACK  OP  CODE  AND  ACC#; 

DO  CASS  (STACK  OP  CODE) ; 

CASEO; 

DO; 

CYCLE  TABLE  INDEX  = 1 ; 

ASSIGN  ACCUMULATOR (ACC#)  TO  FRAME  POINTER; 

B'JD; 

CASEls 

DO; 

CYCLE  TABLE  INDEX  = 1 ; 

ASSIGN  FRAME  POINTER  TO  ACCUMULATOR (ACC#) ; 

END; 

CASES; 

DO; 

DO  LOOP  = 0 TO  2 BY  1; 

INCREMENT  STACK  POINTER ‘BY  1; 

IF  STACK  OVERFLOW  THEN 
DO; 

SETUP  STACK  INTERRUPT  REQUEST; 

RETURN  TO  CALLER; 

END; 

ASSIGN  ACCUMULATOR  (LOOP)  TO  MAIN  MEMORY  (STACK  POINTER); 

END; 

INCREMENT  STACK  POINTER  BY  I; 

CYCLE  TAELS  INDEX  = ?; 

ASSICN  FRAME  POINTER  TO  MAIN  MEMORY (STACK  POINTER) ; 

INCREMENT  STACK  POINTS.  BY  1; 

ASSIGN  CARRY  BIT  AND  ACCUMULATOR (3)  TO  MAIN  MEMORY (STACK  1X)INTER) ; 
-ASSIGN  STACK  POINTER  TO  FRAME  POINTER  AND  TO  ACCUKUL\T0R(3) ; 

END; 


l • t 


CASE3« 

DOj 

CYCLE  TABLE  INDEX  = 6; 

ASSIGN  FRAME  POINTER  TO  STACK  POINTER 5 

ASSIGN  CARRY  BIT  AND:  LOCATION  COUNTER  FROM  MAIN  MEMORY (STACK  POINTER) ; 
DECREMENT  STACK  POINTER  BY  lj 
DC  LOOP  = 3 TO  0 BY  -1; 

ASSIGN  MAIN  M3iCRY( STACK  FOINTER)  TO  ACCUMULATOR(LOCP) ; 

DECREMENT  STACK  POINTER  BY  lj 
END; 

END; 

CASE4| 

DO; 

CYCLE  TABLE  INDEX  = 1 ! 

ASSIGN  ACCUMULVTCR(ACC#)  .TO  STACK  POINTER; 

END ; 

CASE5I 

DO; 

CYCLE  TABLE  INDEX  = 1; 

ASSIGN  STACK  POINTER  TO  ACCUKMULATOR(ACC#)  ; 

END; 

CASE61 

DO; 

CYCLE  TABLE  INDEX  =>  3? 

INCREMENT  STACK  FOINTER  BY  1; 

IF  STACK  OVERFLOW  THEN  SETUP  STACK  INTERRUPT; 

ELSE  ASSIGN  ACCUKULATCR(ACC#)  TO  MAIN  MEMORY (STACK  POINTER); 

END; 

CASE7« 

DO; 

CYCLE  TABLE  INDEX  = 3; 

ASSIGN  MAIN  MEMCRY(STACK  POINTER)  TO  ACCUMULATOR (ACC#) { 

DECREMENT  STACK  POINTER  BY  1; 

END; 

END  CASE; 

RETURN  TO  CALLER; 

END  STACK  OR  MUL  DIV; 
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CPU  FUNCTI ON'S : PROCEDURE; 

~ DECOMPOSE  INSTRUCTION  INTO  OP  CODS,  ACC//,  INTIiRRUFT  SPECIFICATION; 

DO  CASE  (-OP  CODE) ; 

CAS SO j 

IF  INSTRUCTION  IS  "INTERRUPT  31  ABLE"  AND  INTERRUPTS  ARE  NOT 
ENABLED  THEN  IDE.’TIFY  INSTRUCTION  FOR  MAIN  SIMULATOR  LCD?; 

CAS El s 

EXECUTE  "CASE2"  CODS} 

CASE2 1 

DO  CASE  (ACC//)  ; 

INNER_CASSO; 

RESET  BUSY  FLAGS,  DONE  FLAGS , DEVICE  ON  FLACS, 

CURRENT  PRIORITY  MASK  5 
INNER_CASE1 : 

DISABLE  REAL  TIKE  CLOCK; 

INNER_CASE2i 

DO; 

ENABLE  REAL  TIKE  CLOCK; 

INITIALIZE  REAL  TIME  CLOCK  TO  RANDOM  RESET  VALUE; 

END; 

INNER_CASE3 ; 

TERMINATION  CODE  = INVALID  INSTRUCTION  CODE; 

END  CASE; 

CASE3: 

SET  ACCUFiUL.\TOR(ACC/0  BITS  ACCORDING  TO  "INTERRUPT  ACKNOWLEDGE" 
SPECIFICATICNS ; 

CASE4; 

DO; 

ASSIGN  A CCUM ULATOR ( A CC// ) TO  NEW  PRIORITY  MASK; 

IDENTIFY  "MASK  OUT"  INSTRUCTION  FOR  MAIN  SIMULATOR  LOOP; 

END; 

CASE5: 

TERMINATION  CODE  = INVALID  INSTRUCTION  CODE; 

CASE6: 

DO; 


TERMINATION  CODE  = "HALT"  INSTRUCTION  CODE; 

ENABLE  INTERRUPTS; 

RETURN  TO  CALLER; 

END; 

END  CASE; 

ENABLE  OR  DISABLE  INTERRUPTS  ACCORDING  TO  INTERRUPT  SPECIFICATION 
RETURN  TO  CALLER ; 

END  CPU  FUNCTION’S; 


: *V/- . ■> 


io  in:;t«  procedure; 

“ DECOMPOSE  INSTRUCTION  INTO  10  OP  CODE  AND  DEVICE  CODE; 

IF  10  OP  CODE  = ? THEN 
DO; 

SET  SKIP  INDICATOR  ACCORDING  TO  "IOSKIP"  INSTRUCTION  SPECIFICATIONS 
RETURN  TO  CALLER ; 

END; 

IF  10  OP  CODE  IS  A ’’DATA  IN'*  OPERATION  THEN 
DO; 

IF  DEVICE  CODE  = 40  THEN 
DO; 

PERFORM  SIMULATOR  BLOCK  INPUT  ACCORDING  TO  THE 
FOLLOWING  INSTRUCTIONS; 

EFFECTIVE  ADDRESS  = LOCATION  COUNTER; 

INCREMENT  LOCATION  COUNTER  BY  1; 

CALL  FIND  INDIRECT  ADDRESS; 

INPUT  16  WORDS  FROM  SYSIN  ASSIGNING  THEM  TO  MAIN  MEMORY 
STARTING  AT  EFFECTIVE  ADDRESS; 

END; 

ELSE  ICNORE  "DATA  IN  OPERATION  FOR  SIMULATION  PURPOSES; 

END; 

ELSE  IF  10  OP  CODE  IS  A "DATA  OUT"  OPERATION  THEN 
DO; 

IF  DEVICE  CODE  = 41  THEN 
DO; 

PERFORM  SIMULATOR  BLOCK  OUTPUT  ACCORDING  TO  THE 
FOLLOWING  INSTRUCTIONS ; 

EFFECTIVE  ADDRESS  = LOCATION  COUNTER; 

INCREMENT  LOCATION  COUNTER  BY  1; 

CALL  FIND_INDIRSCTADDRSSS ; 

STARTING  AT  EFFECTIVE  ADDRESS  IN  MAIN  MEMORY 
OUTPUT  16  WORDS  TO  SYSPRINT; 

END; 

ELSE  IGNORE  "DATA  OUT*  OPERATION  FOR  SIMULATION  PURPOSES; 

END; 

SET  BUSY  AND  DONE  FLAGS  ACCORDING  TO  SPECIFICATIONS ; 

RETURN  TO  CALLER; 

END  10  INST; 


TRACE  DUMP;  PROCEDURE; 

EXTRACT  10  FORMAT  ACC./  FROM  INSTRUCTION ; 

EXTRACT  COUNT  FROM  BITS  5 TO  9 OF  INSTRUCTION; 

DO  CASS  (ACC//) ; 

CAS  SO  i 
DO; 

IF  TRACE  REQUEST  IS  NOT  ON  THEN 
DO; 

IF  COUNT  = 0 THEN  RETURN  TO  CALLER; 

ASSIGN  LOCATION  COUNT'S  - 1 TO  ACTIVE  TRACE  LOCATION; 

TURN  ON  TRACS  REQUEST; 

TRACS  COUNT  = COUNT; 

RETURN  TO  CALLER; 

END; 

IF  ACTIVE  TQACE  LOCATION’  t LOCATION  COUNTER  - 1 THEN 
RETURN  TO  CALLER; 

DECREMENT  COUNT  BY  1; 

STORE  COUNT  BITS  BACK  INTO  CORRESPONDING  INSTRUCTION  BITS 
AND  RESTORE  MODIFIED  INSTRUCTION  TO  MAIN  MEMORY; 

IF  COUNT  = 0 THEN 

DO;  ACTIVE  TRACE  LOCATION  = 0;  TURN  OFF  TRACE  REQUEST;  END; 

END; 

CASElt 

DO;  ACTIVE  TRACE  LOCATION  = 0;  TURN  OFF  TRACE  REQUEST;  END; 

CASE2i 

PERFORM  DUMP; 

CASE3» 

DO;  PERFORM  DUMP;  TERMINATION  CODE  = DUMP  EXIT  CODE;  END; 

END  CASE; 

RETURN  TO  CALLER; 

END  TRACE_DUKP; 
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APPENDIX  N - DESIGN  COMPREHENSION  TEST 


Name: 


QUIZ 


This  quiz  consists  of  thirty  multiple-choice  questions  and  two  fill-in 
questions.  Please  pace  yourself  so  that  you  can  complete  all  questions 
in  the  time  allowed.  Answer  all  questions.  If  you  don't  know  the  answer 
to  a multiple-choice  question,  eliminate  any  responses  which  are  clearly 
wrong,  and  guess  among  the  remaining  responses. 


tf 


N-l 
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Code: 


DO  NOT  TURN  THE  PAGE  until  you  have  completed  the  work  on  this  page. 
Then  do  not  return  to  this  page. 


List  below  the  names  of  all  the  modules  in  the  simulator.  List  them 
exactly  if  you  can;  if  you  are  unable  to  reproduce  a module  name 
exactly,  then  come  as  close  as  you  can. 


For  each  of  the  multiple-choice  questions  which  follow,  select  the 
one  best  alternative  by  circling  the  letter  which  precedes  it. 

Example:  Most  automobiles  are  powered  by 

(a)  steam  engines. 

§ turbine  engines. 

internal-combustion  engines, 
horses. 

1.  According  to  the  design,  a simulator  entity  called  DEVICE  CODE 

(a)  is  used  to  identify  a peripheral  device  for  I/O  instructions, 

(b)  is  used  to  prevent  some  instruction  classes  from  being 
interpreted  as  I/O  instructions. 

(c)  both  (a)  and  (b). 

(d)  none  of  these. 


Assume  that  the  microNova  simulator  has  been  implemented  in 
accordance  with  the  design  you  have  been  given,  and  that  you  are 
now  debugging  the  simulator.  You  observe  that  the  "Load  Accumulator" 
instruction  works  correctly,  but  the  "Store  Accumulator"  instruction 
does  not.  Which  module  probably  contains  the  error? 

(a)  EXECUTE  INSTRUCTION 

(bj  CPU  FUNCTIONS 

(c) "COMR)TE_EFFECTIVE_ADDRESS 

(d)  MEMORY  REFERENCE  INST 


According  to  the  design,  the  location  counter  is  incremented 

(a)  immediately  before  instruction  fetching. 

(b)  immediately  after  instruction  fetching. 

(c)  immediately  after  instruction  execution. 

(d)  if  and  only  if  the  skip  indicator  is  on. 


4.  The  multiway  branch  specifications  used  in  the  design  specify 
for  an  "out  of  range"  condition  that 

(a)  a precise,  but  variable,  action  is  always  to  be  taken. 

(b)  nothing  is  to  be  done  as  a consequence. 

(c)  an  error  message  is  to  be  printed  and  execution  continued. 

(d)  the  termination  code  is  to  be  set  to  an  "abort"  code. 


5.  How  many  input/output  device  codes  does  the  microNova  allow? 

(a)  16 

(b)  32 

(c)  64 

(d)  128 


m*TZT 


6.  Suppose  the  MEMORY_REFERENCE_INST  module  is  invoked  immediately 

upon  entry  to  the  EXECUTE__IN?TRUCTION  module.  (Assume  the  invocation 
in  the  current  design  is  removed.)  The  implications  are  that 

(a)  redundant  processing  will  occur  but  logical  equivalence  with 
the  current  design  results. 

(b)  logical  equivalence  will  result  with  no  redundant  processing. 

(c)  the  CYCLE  TABLE  INDEX  value  may  be  incorrectly  set. 

(d)  incorrect  simulation  may  result. 


7.  The  manner  in  which  the  current  cycle  count  is  used  in  the  simulator 
will 

(a)  prevent  infinite  loops  in  the  simulator  program. 

(b)  prevent  infinite  loops  in  the  microNova  program. 

(c)  both  (a)  and  (b). 

(d)  none  of  these. 


8.  Suppose  the  simulator  will  eventually  be  used  to  simulate  microNova 
programs  in  object  deck  form  whose  format  is  not  yet  known.  In 
order  to  simulate  these  programs 

(a-),  the  .instruction  execution  module  will  have  to  be  redesigned. 

(b)  simulation  of  interrupts  and  I/O  will  need  revision. 

(c)  simulator  initialization  will  require  revision. 

(d)  the  simulator  main  procedure  will  need  revision. 


9.  If  the  specifications  for  FETCH  INSTRUCTION  and  invocation  of 
EXECUTE_INSTRUCTION  are  interchanged,  the  implications  are  that 
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(c) 


(d) 


logical  equivalence  with  the  current  design  will  result, 
logical  equivalence  with  the  current  design  will  result 
If  simulator  Initialization  is  modified  to  fetch  the  first 
instruction. 

logical  equivalence  with  the  current  design  will  result 
if  all  location  counter  value  references  are  changed  and 
simulator  initialization  is  modified  to  fetch  the  first 
Instruction. 

logical  equivalence  with  the  current  design  will  require 
substantial  redesign  of  several  modules. 


10. 


Suppose  you  are  debugging  the  simulator,  and  you  observe  that 
you  can  successfully  turn  the  trace  on,  that  trace  information  is 
correctly  printed,  but  that  you  cannot  turn  the  trace  off.  Which 
module  probably  contains  the  error? 

(a)  MICRONOVA  SIMULATOR  (main  procedure) 

(b)  EXECUTE  INSTRUCTION 

(c)  TRACE  DUMP 

(d)  10  IN5Y 


I 


y 
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11.  In  the  interrupt  processing  procedure  there  is  a section  of  logic 

Ifor  assigning  a failure  code  to  the  termination  code.  If  this 

logic  is  executed  the  meaning  is  that 

(a)  there  is  an  error  in  the  microNova  program. 

(b)  there  is  an  error  in  the  indirect  address  computation. 

(c)  there  is  an  error  in  the  simulator  program. 

(d)  none  of  these. 


12.  The  failure  code  reference  in  question  11  represents 

(a)  a simulation  of  a microNova  hardware  feature. 

(b)  a simulation  of  a feature  in  the  microNova  interrupt  service 
program. 

(c)  a feature  of  the  simulation  design. 

(d)  all  of  the  above. 


13.  The  current  cycle  count  represents 

(a)  the  number  of  microNova  instructions  simulated. 

(b)  the  index  of  the  main  simulator  loop. 

(c)  an  approximation  to  the  time  of  execution  of  the  simulator 
program. 

(<±)  an  approximation  to  the  time  of  execution  of  the  microNova 
program. 


14.  In  microNova  instructions  which  involve  relative  addressing,  a 
part  of  the  instruction  word  contains  displacement  information. 
How  Is  that  information  represented? 

(a)  one's  complement 

(b)  two's  complement 

(c)  excess-64 

(d)  excess  128 


15.  If  an  effective  address  is  computed  which  is  outside  the  range  of 
addresses  of  the  machine  being  simulated,  the  design 

(a)  clearly  specifies  that  "wraparound"  addressing  results. 

(b)  specifies  that  an  "abort"  termination  code  is  to  be  set. 

(c)  is  not  very  precise  about  the  action  to  be  taken. 

(d)  none  of  these. 


i 

■i 
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16.  According  to  the  design,  the  procedure  labelled  FIND_INDIRECT_ADDRESS 

(a)  may  be  used  to  follow  a chain  of  indirect  addresses. 

(b)  is  always  used  to  retrieve  the  contents  of  exactly  one  word; 
thus  a chain  of  indirect  addresses  is  processed  by  repeated 
invocation  of  the  named  procedure. 

(c)  may  not  be  invoked  until  address  modification  due  to  indexing 
(if  any)  has  been  performed. 

(d)  could  easily  be  absorbed  into  the  design  of  the  module  for 
memory  reference  instructions. 


17.  The  cycle  table  index  is  used  for 

(a)  indexing  a table  used  for  recording  statistics  about  the 
simulation  process. 

(b)  retrieving  approximate  instruction  execution  times. 

(c)  a loop  index  during  cycle  table  processing. 

(d)  none  of  these. 


18.  The  index  bits  for  memory  reference  instructions,  interpreted  as  a 
nonnegative  integer  value  are  used  in  the  simulator  design 

(a.)  as  an  index  to  a vector  of  size  four  used  for  simulating  the 
accumulators  used  as  index  registers. 

(b)  to  modify  the  signed  displacement. 

(c)  as  a decrement  value  for  auto  index  word  references. 

(d)  to  determine  an  "intermediate"  effective  address  just  before 
possible  modification  due  to  indirect  addressing. 


19.  If  the  skip  indicator  is  removed  from  the  design,  then  it  will  be 
necessary  to  make  which  one  of  the  following  additional  changes  in 
the  design? 

(a)  immediately  after  returning  from  the  instruction  execution 
routine,  the  instruction  just  executed  should  be  tested  to 
determine  whether  the  location  counter  is  to  be  incremented. 

(b)  modify  the  design  for  simulation  of  the  JUMP  instruction. 

(c)  whenever  an  instruction  is  fetched,  it  should  be  tested 
immediately  for  a skip  specification;  if  one  is  present,  the 
location  counter  should  be  Incremented. 

(d)  wherever  the  skip  indicator  Is  turned  on  in  the  current  design, 
replace  that  with  a location  counter  increment. 


20.  Suppose  you  are  debugging  the  simulator,  and  you  observe  that  "Trap" 
instructions  work  correctly,  and  that  "Load  Accumulator"  and  "Jump" 
instructions  do  not  work  when  they  use  indirect  addressing,  but 
succeed  otherwise.  Which  module  probably  contains  the  error? 


(a)  FINDJNDIRECT  ADDRESS 

(b)  COMPUTE_EFFECTIVE  ADDRESS 

(c)  MEMORY  REFERENCE  TNST 

(d)  EXECUTljNSTRUCTlON 


21.  For  purposes  of  illustration,  suppose  there  are  25  instructions  in 
the  microNova  instruction  set.  An  alternative  way  of  designing  the 
simulator  might  be  to  provide  a procedure  to  determine  which  one  of 
the  25  instructions  is  to  be  simulated  and  then  have  a 25-way 
branch  to  25  different  sets  of  code  which  perform  the  actual 
simulation.  Compared  to  the  design  given  to  you,  the  implications 
of  such  an  approach  are  that 

(a)  substantial  duplication  of  code  might  result  unless  such 
duplication  is  removed  to  subprograms. 

(b)  modification  of  the  resulting  simulator  would  be  simpler, 
although  initial  implementation  would  take  longer. 

(c)  execution  time  for  the  simulator  would  be  significantly 
shorter. 

(d)  none  of  these. 


22.  The  purpose  of  the  CPU_FUNCTIONS  module  is  to 

(a)  assign  values  to  CPU  registers  from  a special  input  device. 

(b)  simulate  instructions  associated  with  interrupts. 

(c)  simulate  the  microNova  CPU. 

(d)  none  of  these. 

23.  The- outermost  loop  of  the  simulator  is  controlled  by 

(a)  the  Instruction  cycle  count. 

(b)  pending  Interrupts. 

(c)  a loop  index. 

(d)  a termination  code. 


24.  The  tracing  and  dumping  features  may  be  used  for 

(a)  debugging  microNova  programs. 

(b)  debugging  the  simulator. 

(c)  both  (a)  and  (b). 

(d)  none  of  these. 


25.  If  an  interrupt  is  to  be  simulated,  the  interrupt  processing 
procedure 

(a)  fetches  the  first  instruction  of  the  interrupt  service  routine. 

(b)  decrements  a value  in  simulated  memory  by  one. 

(c)  Invokes  the  procedure  for  computation  of  effective  addresses 
for  proper  determination  of  the  location  counter  value. 

(d)  none  of  the  above. 
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26.  The  function  of  the  microNova  shifter,  when  executing  a left  shift 
operation,  can  best  be  described  as  a 

(a)  16-bit  end-around  shift. 

(b)  16-bit  no-end-around  shift. 

(c)  17-bit  end-around  shift. 

(d)  17-bit  no-end-around  shift. 


27.  Suppose  the  design  is  to  be  revised  so  the  body  of  the  outer  loop 
of  the  simulator  is  shorter  but  the  entire  design  is  logically 
equivalent  to  the  original  design.  Without  changing  any  other 
procedures,  it  would  be  possible  to  put  some  of  the  processing  in 

(a)  the  simulator  initialization  module. 

(b)  the  instruction  execution  module. 

(c)  the  module  which  computes  effective  addresses. 

(d)  any  of  the  above. 


28.  The  purpose  of  the  simulator  initialization  procedure  is  to 

(a)  load  a microNova  machine  language  program  into  simulated 
memory. 

(bj  initialize  simulated  machine  flags  and  registers. 

(c)  initialize  simulator  program  control  variables. 

(d)  all  of  the  above. 


29.  Suppose  you  are  debugging  the  simulator,  and  you  observe  that 
"Interrupt  Enable"  instructions  do  not  work  correctly.  Which 
module  probably  contains  the  error? 

(a)  CPU  FUNCTIONS 

(b)  PROCESS  INTERRUPT 

(c)  10  INST 

(d)  EXECUTE  INSTRUCTION 


30.  The  maximum  number  of  microNova  instructions  simulated  In  one 
pass  through  the  body  of  the  outermost  simulator  loop  is 

(a)  1. 

(b)  2. 

(c)  3. 

I (d)  4 or  more. 
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Shown  on  the  next  page  Is  an  adjacency  matrix.  The  matrix  is  to  be 
used  to  indicate  which  modules  invoke  which  other  modules  directly. 

Rows  represent  calling  modules,  while  columns  represent  called  modules. 
You  are  to  enter  an  "X"  in  each  cell  which  represents  a calling-module/ 
called-module  relation  in  the  simulator  design.  Be  sure  you  understand 
the  example  below  before  you  fill  in  the  matrix  on  the  next  page. 

Example:  Suppose  that  a design  contains  modules  A,  B,  C,  and  D. 

Suppose  that  A calls  B and  C,  and  that  B calls  C and  0.  Then  the  matrix 
for  this  design  has  the  form: 


Called  module 


J; 


PROCESS  INTERRUPT 


wrr — 


APPENDIX  0 - TIME  EXPENDITURE  QUESTIONNAIRE,  IMPLEMENTATION  PHASE 
WEEKLY  TIME  QUESTIONNAIRE  (Implementation  Phase) 

Name : 


Date: Code: 

Please  record  the  time  you  have  spent  on  your  class  project  in  the  last  week, 
excluding  time  spent  in  class.  Record  time  in  hours  and/or  minutes  by 
category. 

hrs  : min 

Time  spent  in: 

Initial  code  preparation  : 

Modification  of  existing  code  : 

Keypunching  : 

Preparation  of  test  cases  : 

Execution  and  interpretation  of  test  runs : 

Preparation  of  documentation  : 

Just  thinking  about  project  : 

Other  (indicate  nature  of  activity) 


This  information  is  solely  for  experimental  purposes;  your  responses  will  in 
no  way  affect  your  course  grade.  Please  be  as  accurate  as  possible. 
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